What's the speed secret to "Build Your Own Atom-based NAS - Part 2"?

[milo]

Hi All,

This is my first post but I have done a lot of searches, plus testing of various software but am now just stumped, so I must ask..... What is the secret to getting the speeds shown in Build Your Own Atom-based NAS - Part 2?

I was very inspired by the article and went out to purchase some gear. I ended up with the following:
- Intel BOXD945GCLF2 Atom 330 ITX Motherboard/CPU Combo
- Samsung Spinpoint EcoGreen F2 HD103SI 1TB Hard Drive (x2 for RAID1)
- Samsung 1 GB DDR2 PC6400 800MHz Memory - M378T2863QZS-CF7
- Slim DVD/CD-ROM IDE drive
- IN-WIN CASE BM648.AD120TBL MINI ITX MINI TOWER Case

I have used both a IDE to CF Adaptor with a 128MB CF card for OS when using FreeNas and have also used an old 10GB Travelstar 2.5" IDE drive with both FreeNas and Ubuntu.

I think this set-up is pretty close to that in the article listed above but my throughput is MUCH slower. In the start I was using FreeNas i386 0.69.2 with the drivers added for the on-board GBe. At that time I was seeing 5-8 MBytes/s when transferring from my XP Pro system to the Nas via wire GBe. I then jumped to Ubuntu (tried both 8.04 LTS & 9.04 64-bit versions) but had a problem getting the Webmin to work correctly. After setting up the RAID1 via the command line, I got a small increase with Ubuntu to 6-10 MBytes/s. Using the same computer running Vista 64-bit I was at 12-18 MB/s.

Depressed and frustrated with the Webmin problems, I when back to FreeNas and tried "Tuning". Tuning helped a little but didn't do much so I tried jumping to the 0.7.4706 nightly build of FreeNas which got me up to 15-25 MBytes/s. On the 0.7 build tuning doesn't seem to make much of a difference and I also am not seeing too much of a difference between XP pro and Vista 64.

My throughput has gotten a lot better than the start of this project but still not near the 35-45 MB/s write and 55-60 MB/s read speeds seen in the article. Also, my read and write speeds seem to be about the same. Any thoughts on what the problems might be? Things to turn off in bios or to turn on? etc.

Thanks,

Milo

P.S. One last thought, my SATA controller shows up in FreeNas as SATA300 but both my drives are listed as SATA150 even though they are SATA300 drives. Is this normal?

[00Roush]

My first thought would be the fact that you are using the onboard network card. In the article that was disabled and a Intel PCI network card was used which is most likely much better supported under FreeNAS and Ubuntu.

I would recommend doing some tests with Iperf to see if your network performance is good. Then move on to file transfer speeds.

On FreeNAS make sure you change the send and receive buffers to 65536 under the CIFS settings.

My last install of Webmin on Ubuntu Server 8.04 ran into some problems as well. Here is how I got it to work... Add deb http://download.webmin.com/download/repository sarge contrib to your sources.list file as recommended in the Atom article. Then execute the following commands:
cd /root
sudo wget http://www.webmin.com/jcameron-key.asc
sudo apt-key add jcameron-key.asc
sudo apt-get update
sudo apt-get install webmin

Hopefully that should get it installed for you. Also I would generally recommend trying the 32 bit Ubuntu Server versions first to limit the chance of problems.

I usually disable serial ports, parallel ports and onboard sound in the BIOS on any computer I plan to use as a server.

If you are looking for the best SMB/SAMBA speeds make sure you are testing with Vista SP1 on the client.

00Roush

[milo]

Thanks for the tips. I believe the network card in the article was only a 10/100 on the single-core Atom board. That was one of the main reasons I went with the dual-core Atom instead. I figured a dual core couldn't hurt and I'd rather have GBe built-in. Although there still might be some performance differences between the Intel PCI card vs my Realtek built-in Ethernet. I'm not familiar with Iperf so it looks like I have more reading and testing to do. Thanks for pointing me in the right direction.

As far as buffers, I played with the buffers under the tuning option in System|Advanced|sysctl.conf but never tried (or noticed) those setting under the CIFS setting before. I am a newbie to this kind of stuff so it looks like I have a few more things to try with FreeNas before I give up. I am also a bit surprised at the jump I saw with 0.7 vs 0.69 so maybe there is hope.

Thanks for the tips on Webmin also. I put the 10G drive back into the system today to go back to Ubuntu but haven't reconnected it yet and still have FreeNas running off the CF card for now. I'll see what I can do with FreeNas with the new info and then compare Ubuntu again.

Thanks

[roamer]

Hello,

I build a similar configuration after read this nice article about Ubuntu on Atom-platform.

My config:

Intel D945GCLF2 board (with Realtek 8111C onboard)
2x 1TB WD EADS hdd as mirror
2 GB RAM
4GB CompactFlash for Ubuntu
and Ubuntu 8.10 server

The write performance from a PC with Vista (core2duo,4GB RAM) was between 50-60 MB/s with a bigfile (>2,5GB).



- Have you tested the speed with ftp instead of samba ?
( you have to install a ftp-server on the NAs e.g pureFTPd )

- login on your nas and start the 'top' tool during the file transfer
is there any process that is consumpting a lot of CPU or RAM resources?

-Are you disks in the nas are ok - can you locally on the nas copy such big files from one directory to another?
have simply tested speed of a single disk with ubuntu
e.g hdparm -tT /dev/sdc

You should also check on your NAS the logfiles in the directory /var/log for errors. during the filetransfer Especially in the messages files.

[thiggins]

Thanks for the tips. I believe the network card in the article was only a 10/100 on the single-core Atom board.
The onboard NIC was disabled and an Intel PRO/1000 PCI used.

[milo]

roamer, thanks for the info. It looks like your system really flies.

My test computer is a Asus G50VT-A2 laptop and I have been using files in the 1-1.25 GB and 4-4.5 GB range during testing. It is also a core2duo (2.53Ghz) with 4GB ram (well only 3GB when using XP Pro).

I did not try FTP using Ubuntu but have tried it in FreeNAS. In FreeNAS 0.69 the FTP speeds where a bit faster but now with version 0.7 the Samba and FTP speeds are about the same.

Last night I played with the buffers (and other settings) under CIFS as 00Roush mentioned and was able to get FreeNAS running between 28-35 MB/s. (Thanks 00Roush!) I have checked the logs and nothing seems obviously wrong but then again it just might not be obvious to me.

I haven't tried NAS to NAS file transfers or the other things you mentioned so I'll look into those too.

Is there a reason you chose 8.10 server over 8.04 LTS? Also, are you running 32-bit or 64-bit? I'm about to jump back to Ubuntu tonight.

EDIT: I saw in another post of your that you are running 32-bit, but I still wonder why 8.10 over 8.04 LTS or 9.04?

[milo]

The onboard NIC was disabled and an Intel PRO/1000 PCI used.

Hi thiggins,

Sorry if my post came across wrong. I knew the on-board was disabled and a GbE PCI card was used. I was simply trying to point out the differences between the two on-board NICs and the possible / probable reason a PCI card was used instead of correcting the driver for the non-working on-board NIC.

Thanks for the GREAT articles on the Atom based NAS. I had purchased a Linksys WRT610n router in hopes of using it but I never check this site before my purchase and only realized how snail slow it was, afterwards. Now I'm simply using it for GbE and N wireless. :(

[roamer]

Is there a reason you chose 8.10 server over 8.04 LTS? Also, are you running 32-bit or 64-bit? I'm about to jump back to Ubuntu tonight.

EDIT: I saw in another post of your that you are running 32-bit, but I still wonder why 8.10 over 8.04 LTS or 9.04?

I prefer 32bit
- because less then 4GB RAM is used
- there were some trouble in past with Realtek NIC and 64bit e.g here (https://bugs.launchpad.net/ubuntu/%2Bsource/linux/%2Bbug/240470), I want to bypass such problems

Why not 8.04 -> It's older then 8.10
Why not 9.04 -> It was not released
Why not still upgraded -> don't toch a running system :D

[milo]

Thanks for the info, once again.

I still have FreeNAS running for now and just tried the NAS to NAS transfer. I have to say that I'm really confused with the results. I started by making two directories on the NAS drive via windows. I then transferred a 4.35GB file from my computer into one of the folder in 172 seconds (about 25 MB/s). I then tried using windows to copy the file from one folder to another on the NAS and it took forever! Five minutes into the transfer I was only half way through so I canceled the copy. I then sshed into the NAS and tried again with a cp command. This time it took 235 second or about 18MB/s. Why would the NAS to NAS take longer than the computer to NAS transfer. Is it due to my Raid 1 set-up? Basically reading and writing at the same time. Even if this is it, I figure the drive would work faster than the network(?). I'm confused and done for the night.

I think I'm going to give Ubuntu 9.04 32-bit server a try tomorrow. Thanks again.

[roamer]

Thanks for the info, once again.

I still have FreeNAS running for now and just tried the NAS to NAS transfer. I have to say that I'm really confused with the results. I started by making two directories on the NAS drive via windows. I then transferred a 4.35GB file from my computer into one of the folder in 172 seconds (about 25 MB/s). I then tried using windows to copy the file from one folder to another on the NAS and it took forever! Five minutes into the transfer I was only half way through so I canceled the copy. I then sshed into the NAS and tried again with a cp command. This time it took 235 second or about 18MB/s. Why would the NAS to NAS take longer than the computer to NAS transfer. Is it due to my Raid 1 set-up? Basically reading and writing at the same time. Even if this is it, I figure the drive would work faster than the network(?). I'm confused and done for the night.

I think I'm going to give Ubuntu 9.04 32-bit server a try tomorrow. Thanks again.


Give Ubuntu a try, if you have read my previous posts in another thread then you know that I have also not reached more the 25MB/s with FreeNAS.

--------------
To copy a file via windows client from one directory on the NAS to another directory on the NAS is not a good idea.
That means your are transfering the file from NAS -> PC -> NAS and for shure the LAN connection is busy in both directions.


My suggestion was
- login into the NAS via ssh with e.g. putty and use the 'cp' command to copy a file locally from one disk to another. In this situation the LAN is not involved and it can't be the bottleneck. This steps should be executed without raid configuration from oone single disk to another, if you have still low throughput, you should check yor HW e.g. SATA cable.

If everything works fine locally
- install the ftp-server on the NAS
Transfer files from your PC to the NAS via ftp. with e.g. winscp
With this step the LAN is involved, but not SAMBA.
The troughput should be even higher then with smb/cifs shares, because the ftp protocoll has not so much overhead.
If you have still low throughput, you should check yor HW e.g. LAN-cable.
And it's not wrong to login into the NAS via ssh and start the 'top' tool on the NAS. There you can see which processes consumes CPU-load.

[00Roush]

Thanks for the tips. I believe the network card in the article was only a 10/100 on the single-core Atom board. That was one of the main reasons I went with the dual-core Atom instead. I figured a dual core couldn't hurt and I'd rather have GBe built-in. Although there still might be some performance differences between the Intel PCI card vs my Realtek built-in Ethernet. I'm not familiar with Iperf so it looks like I have more reading and testing to do. Thanks for pointing me in the right direction.

I wanted to get back to you about Iperf... http://en.wikipedia.org/wiki/Iperf It basically is used to measure the best case TCP/IP throughput your network is capable of. This can help you to find out if your network or network card is a bottleneck. This thread might help you figure out how to use it... http://forums.smallnetbuilder.com/showthread.php?t=1118 Now a program like HDTach (windows) lets you know what your best case disk throughput is. For Ubuntu you could use hdparm -t /dev/sda to give you an idea of disk throughput as well. Testing with these programs is usually where I would start when trying to track down where a bottleneck might be.

00Roush

[milo]

00Roush, Thanks for the detailed information. After your first post I googled Iperf and found the Wikipedia page but wasn't ready to try that yet. Last night I reinstalled Ubuntu and used 9.04 32-bit server this time. Using your info, I was able to get webmin installed (with the cert) and it seems to work better this time. At this point I only have Samba and Ssh running and have the two hard drives partitioned independently (not in Raid). My transfer rates from my computer (XP pro) to NAS are still in the 28-30 MB/s range. I then tested a drive to drive copy on the NAS using ssh and got 60 MB/s. To me this still seems very low for a drive to drive copy with these drives.

I will look into using the hdparm tool and see if I can figure out what's happening. I have ran the "top" command as suggested but did not find anything hogging resources. So I think it might have something to do with the drive setup. I still wonder why the SATA300 controller is located but the drives are setup/showing SATA150 when they are really SATA300 drives. Or could it simply be the fact that I only have 1GB of ram rather than 2GB.(?) I was even thinking out trying an install of windows xp or windows 7 RC just to test the transfer speeds with that setup.

Thank you all for the help! As I dig further and learn more, I'll make sure to give updates.

[milo]

UPDATE: OK so I have done some more testing on Ubuntu and am now confused. Using hdparm I ran the following commands and got the following results:

> hdparm /dev/sdb
/dev/sdb:
IO_support = 0 (default)
readonly = 0 (off)
readahead = 256 (on)
geometry = 56065/255/63, sectors = 1953525168, start = 0

> hdparm -Tt /dev/sdb
/dev/sdb:
Timing cached reads: 1208 MB in 2.00 seconds = 604.10 MB/sec
Timing buffered disk reads: 322 MB in 3.02 seconds = 106.76 MB/sec

> hdparm /dev/sdc
/dev/sdc:
IO_support = 0 (default)
readonly = 0 (off)
readahead = 256 (on)
geometry = 56065/255/63, sectors = 1953525168, start = 0

> hdparm -Tt /dev/sdc
/dev/sdc:
Timing cached reads: 1188 MB in 2.00 seconds = 594.18 MB/sec
Timing buffered disk reads: 322 MB in 3.02 seconds = 106.77 MB/sec

If I read this correctly, I have getting 106 MB/s reads on both drives. If this is correct I don't understand why my drive to drive copies are only at 60 MB/s. I also tried to change the IO-support to 32-bit mode but got an error.

Is there something I'm missing?

Thanks

Milo

[roamer]

UPDATE: OK so I have done some more testing on Ubuntu and am now confused. Using hdparm I ran the following commands and got the following results:

> hdparm /dev/sdb
/dev/sdb:
IO_support = 0 (default)
readonly = 0 (off)
readahead = 256 (on)
geometry = 56065/255/63, sectors = 1953525168, start = 0

> hdparm -Tt /dev/sdb
/dev/sdb:
Timing cached reads: 1208 MB in 2.00 seconds = 604.10 MB/sec
Timing buffered disk reads: 322 MB in 3.02 seconds = 106.76 MB/sec

> hdparm /dev/sdc
/dev/sdc:
IO_support = 0 (default)
readonly = 0 (off)
readahead = 256 (on)
geometry = 56065/255/63, sectors = 1953525168, start = 0

> hdparm -Tt /dev/sdc
/dev/sdc:
Timing cached reads: 1188 MB in 2.00 seconds = 594.18 MB/sec
Timing buffered disk reads: 322 MB in 3.02 seconds = 106.77 MB/sec

If I read this correctly, I have getting 106 MB/s reads on both drives. If this is correct I don't understand why my drive to drive copies are only at 60 MB/s. I also tried to change the IO-support to 32-bit mode but got an error.

Is there something I'm missing?

Thanks

Milo

read is always faster then write...

your values quite normal and 60MB/s write is not wrong for a single disk.

[milo]

read is always faster then write...

your values quite normal and 60MB/s write is not wrong for a single disk.

Thanks for the info and confirming these values look OK. My network read and writes are at the same speed and about half the disk to disk speed. Therefore I assume the problem must be something to do with the network.

Time to jump into Iperf and see what I can find.

Thanks again for everyone's help. I thought I knew a bit about this stuff BUT I sure feel like a newbie trying to get this system going. I guess my bit of networking and Linux knowledge is just enough to get me in trouble. :)

[roamer]

just install 'pure-ftpd' on your nas, before you starting with creation of raid...
Check the throughput via ftp from PC to your nas with a single disk.
This step takes only 2minutes and you know what's on.

[00Roush]

read is always faster then write...

your values quite normal and 60MB/s write is not wrong for a single disk.

I tend to disagree with that. Write speeds should be very close to read speeds if those disks are empty. Now with journaling file systems like ext3 write speeds tend to be a little lower but not 40 MB/sec slower.

Milo, what program did you use to do your copy between disks? Also what file system are you using?

00Roush

[milo]

I tend to disagree with that. Write speeds should be very close to read speeds if those disks are empty. Now with journaling file systems like ext3 write speeds tend to be a little lower but not 40 MB/sec slower.

Milo, what program did you use to do your copy between disks? Also what file system are you using?

00Roush

I am currently running Ubuntu 9.04 32-bit server with the ext3 file system on all the drives (OS Disk 10GB, Disk1 1TB, Disk2 1TB). Both the storage (1TB) drives are basically empty other than one or two ~4GB files.

I used Putty from my XP laptop to ssh into the server and then ran the following command for the drive to drive copy; "cp /mnt/Disk1/test.mpg /mnt/Disk2/test.mpg". This file was 4.35GB in size and resulted in about 60MB/s from the time I hit enter to the time the cursor reappeared on the screen (~72sec).

Tonight a ran several tests with Iperf and Jperf. I set up Iperf on the server running as a server and used Jperf on my laptop. I ran all my tests for 10 seconds, reporting every second with MByte output using port 5001. Using the default TCP setting (nothing checked) I got the following:

bin/iperf.exe -c 192.168.1.104 -P 1 -i 1 -p 5001 -f M -t 10
------------------------------------------------------------
Client connecting to 192.168.1.104, TCP port 5001
TCP window size: 0.01 MByte (default)
------------------------------------------------------------
[1912] local 192.168.1.103 port 1061 connected with 192.168.1.104 port 5001
[ ID] Interval Transfer Bandwidth
[1912] 0.0- 1.0 sec 44.3 MBytes 44.3 MBytes/sec
[1912] 1.0- 2.0 sec 37.3 MBytes 37.3 MBytes/sec
[1912] 2.0- 3.0 sec 45.8 MBytes 45.8 MBytes/sec
[1912] 3.0- 4.0 sec 46.6 MBytes 46.6 MBytes/sec
[1912] 4.0- 5.0 sec 47.1 MBytes 47.1 MBytes/sec
[1912] 5.0- 6.0 sec 47.2 MBytes 47.2 MBytes/sec
[1912] 6.0- 7.0 sec 47.4 MBytes 47.4 MBytes/sec
[1912] 7.0- 8.0 sec 45.1 MBytes 45.1 MBytes/sec
[1912] 8.0- 9.0 sec 46.6 MBytes 46.6 MBytes/sec
[1912] 9.0-10.0 sec 46.9 MBytes 46.9 MBytes/sec
[1912] 0.0-10.0 sec 454 MBytes 45.3 MBytes/sec
Done.

I was able to tweak my "Buffer Length" and "TCP Window Size" setting to greatly improve my throughput. The best combination (after many rounds of testing) was using 64 KBytes for Buffer Length and 32 KBytes for TCP Window Size, which gave me the following results:

bin/iperf.exe -c 192.168.1.104 -P 1 -i 1 -p 5001 -w 32.0K -l 64.0K -f M -t 10
------------------------------------------------------------
Client connecting to 192.168.1.104, TCP port 5001
TCP window size: 0.03 MByte
------------------------------------------------------------
[1912] local 192.168.1.103 port 1064 connected with 192.168.1.104 port 5001
[ ID] Interval Transfer Bandwidth
[1912] 0.0- 1.0 sec 108 MBytes 108 MBytes/sec
[1912] 1.0- 2.0 sec 107 MBytes 107 MBytes/sec
[1912] 2.0- 3.0 sec 108 MBytes 108 MBytes/sec
[1912] 3.0- 4.0 sec 109 MBytes 109 MBytes/sec
[1912] 4.0- 5.0 sec 105 MBytes 105 MBytes/sec
[1912] 5.0- 6.0 sec 109 MBytes 109 MBytes/sec
[1912] 6.0- 7.0 sec 109 MBytes 109 MBytes/sec
[1912] 7.0- 8.0 sec 109 MBytes 109 MBytes/sec
[1912] 8.0- 9.0 sec 108 MBytes 108 MBytes/sec
[1912] 9.0-10.0 sec 108 MBytes 108 MBytes/sec
[1912] 0.0-10.0 sec 1080 MBytes 108 MBytes/sec
Done.

My problem is that I have tried to use regedit to make these values my defaults but can't seem to get them to stick. I tried setting TcpWindowSize, DefaultReceiveWindow, DefaultSendWindow in the registry but they didn't seem to change anything. :(

I also tried to install FTP and test but haven't been able to get all the permissions sorted out to allow me to write to my 1TB drives.

Thanks for the help,

Milo

[00Roush]

I am currently running Ubuntu 9.04 32-bit server with the ext3 file system on all the drives (OS Disk 10GB, Disk1 1TB, Disk2 1TB). Both the storage (1TB) drives are basically empty other than one or two ~4GB files....


You shouldn't have to make any changes to the registry... The window size is auto tuned in Windows and to my knowledge also in Ubuntu. You should only have to set buffer size or window size in your tests. I generally use this command line iperf -w 64k -c 192.168.0.2 -r as the results in Mbps make it easier for me to see the network speed. Also this will test both directions.

00Roush

[milo]

I was going by this (http://www.smallnetbuilder.com/content/view/30418/235/) article when trying to adjust the setting and edit the registry, as the article talks about editing the registry for the TCP Window Size. TCP Windows Size made a bit of difference in my reading (from 45 to 65 or so) but the real impact was adjusting the Buffer Length. I then checked this (http://support.microsoft.com/kb/314053/) MS regedit guide but wasn't able to figure out exactly which parameters corresponded to Buffer Length in Jperf.

I will run a few more tests tonight with the Mbps output format and I'm also going to try and get FTP working.

My hope really was lying with the tuning of either the hard drives or networking in Ubuntu. When I had FreeNAS tuned, I was getting the same transfer rates between FTP & Samba. Those transfer rates were also very close to what I am currently getting with Ubuntu.

[00Roush]

I was going by this (http://www.smallnetbuilder.com/content/view/30418/235/) article when trying to adjust the setting and edit the registry, as the article talks about editing the registry for the TCP Window Size. TCP Windows Size made a bit of difference in my reading (from 45 to 65 or so) but the real impact was adjusting the Buffer Length. I then checked this (http://support.microsoft.com/kb/314053/) MS regedit guide but wasn't able to figure out exactly which parameters corresponded to Buffer Length in Jperf.

I will run a few more tests tonight with the Mbps output format and I'm also going to try and get FTP working.

My hope really was lying with the tuning of either the hard drives or networking in Ubuntu. When I had FreeNAS tuned, I was getting the same transfer rates between FTP & Samba. Those transfer rates were also very close to what I am currently getting with Ubuntu.

To my knowledge buffer length and window size in Iperf basically do the same thing. Now with Iperf the program specifically sets the window size based upon either the default or what you specify. With all other TCP/IP traffic Windows controls the window size. The window size setting in the registry sets the maximum possible size allowed. This does not mean that Windows will actually use that size. You can use a program like Wireshark or Microsoft Network Monitor to see the window sizes being used. In my testing I have usually seen it at 60k-64k for file copies across my LAN. If iperf is showing that your network is capable of over 100 MB/sec at a 64k window size I would say your network is good to go and no network setting changes are necessary.

Next up would be testing all of the disks involved to find out if they are the bottleneck. You already tested your drives on your server but what about the client? Also you had mentioned that your drives are showing up as SATA150 instead of SATA300. Could you check that your drives do not have a jumper set that has them in a SATA150 compatibility mode? If you really wanted you could also use Iozone to test your hard drives. On Ubuntu you should be able to install it by typing sudo apt-get install iozone3.

For reference here are my Iperf results:
C:\iperf>iperf -c 192.168.0.2 -P 1 -i 1 -p 5001 -f M -t 10
------------------------------------------------------------
Client connecting to 192.168.0.2, TCP port 5001
TCP window size: 0.01 MByte (default)
------------------------------------------------------------
[132] local 192.168.0.3 port 50788 connected with 192.168.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[132] 0.0- 1.0 sec 49.9 MBytes 49.9 MBytes/sec
[132] 1.0- 2.0 sec 50.1 MBytes 50.1 MBytes/sec
[132] 2.0- 3.0 sec 49.7 MBytes 49.7 MBytes/sec
[132] 3.0- 4.0 sec 49.9 MBytes 49.9 MBytes/sec
[132] 4.0- 5.0 sec 50.0 MBytes 50.0 MBytes/sec
[132] 5.0- 6.0 sec 47.0 MBytes 47.0 MBytes/sec
[132] 6.0- 7.0 sec 49.5 MBytes 49.5 MBytes/sec
[132] 7.0- 8.0 sec 49.7 MBytes 49.7 MBytes/sec
[132] 8.0- 9.0 sec 49.6 MBytes 49.6 MBytes/sec
[132] 9.0-10.0 sec 49.3 MBytes 49.3 MBytes/sec
[132] 0.0-10.0 sec 495 MBytes 49.5 MBytes/sec


C:\iperf>iperf -c 192.168.0.2 -P 1 -i 1 -p 5001 -w 32.0K -l 64.0K -f M -t 10
------------------------------------------------------------
Client connecting to 192.168.0.2, TCP port 5001
TCP window size: 0.03 MByte
------------------------------------------------------------
[132] local 192.168.0.3 port 50789 connected with 192.168.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[132] 0.0- 1.0 sec 113 MBytes 113 MBytes/sec
[132] 1.0- 2.0 sec 112 MBytes 112 MBytes/sec
[132] 2.0- 3.0 sec 112 MBytes 112 MBytes/sec
[132] 3.0- 4.0 sec 112 MBytes 112 MBytes/sec
[132] 4.0- 5.0 sec 112 MBytes 112 MBytes/sec
[132] 5.0- 6.0 sec 112 MBytes 112 MBytes/sec
[132] 6.0- 7.0 sec 112 MBytes 112 MBytes/sec
[132] 7.0- 8.0 sec 112 MBytes 112 MBytes/sec
[132] 8.0- 9.0 sec 112 MBytes 112 MBytes/sec
[132] 9.0-10.0 sec 112 MBytes 112 MBytes/sec
[132] 0.0-10.0 sec 1123 MBytes 112 MBytes/sec


C:\iperf>iperf -w 64k -c 192.168.0.2 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.0.2, TCP port 5001
TCP window size: 64.0 KByte
------------------------------------------------------------
[156] local 192.168.0.3 port 50853 connected with 192.168.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[156] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec
[168] local 192.168.0.3 port 5001 connected with 192.168.0.2 port 1311
[ ID] Interval Transfer Bandwidth
[168] 0.0-10.0 sec 1.10 GBytes 944 Mbits/sec


00Roush

[milo]

To my knowledge buffer length and window size in Iperf basically do the same thing. Now with Iperf the program specifically sets the window size based upon either the default or what you specify. With all other TCP/IP traffic Windows controls the window size. The window size setting in the registry sets the maximum possible size allowed. This does not mean that Windows will actually use that size. You can use a program like Wireshark or Microsoft Network Monitor to see the window sizes being used. In my testing I have usually seen it at 60k-64k for file copies across my LAN. If iperf is showing that your network is capable of over 100 MB/sec at a 64k window size I would say your network is good to go and no network setting changes are necessary.

Next up would be testing all of the disks involved to find out if they are the bottleneck. You already tested your drives on your server but what about the client? Also you had mentioned that your drives are showing up as SATA150 instead of SATA300. Could you check that your drives do not have a jumper set that has them in a SATA150 compatibility mode? If you really wanted you could also use Iozone to test your hard drives. On Ubuntu you should be able to install it by typing sudo apt-get install iozone3.

00Roush


Thanks testing at MBytes/s and the info. I have checked the drives for jumpers and have even downloaded the Samsung Test/Configuration LiveCD to check the drives and ensure they are at SATA300, which they are. It's only during the boot messages that I see Ubuntu has them listed at SATA150.

Last night, I removed all my registry edits on my XP system and saw that it really made no difference. So I took a different direction all together and install XP Pro SP3 onto the server. I didn't have the copy activated but I don't think that would matter for this testing. I repeated my Laptop to Server tests and found my speeds dropped to about 20MB/s. Next, I tried drive to drive tests (on the now XP Pro server) and found that my speeds increased to 80-90MB/s. This makes me believe that my problem is somewhere in the drives using Ubuntu, or at least that Ubuntu could do better.

Would the speed of the OS drive matter? Currently it's a rather old laptop drive I pulled from a scrap laptop. Also, I will also use HDtach to test my laptop's hard drives but I think they should be good since they are fairly new.

Thanks

[00Roush]

Thanks testing at MBytes/s and the info. I have checked the drives for jumpers and have even downloaded the Samsung Test/Configuration LiveCD to check the drives and ensure they are at SATA300, which they are. It's only during the boot messages that I see Ubuntu has them listed at SATA150.

Last night, I removed all my registry edits on my XP system and saw that it really made no difference. So I took a different direction all together and install XP Pro SP3 onto the server. I didn't have the copy activated but I don't think that would matter for this testing. I repeated my Laptop to Server tests and found my speeds dropped to about 20MB/s. Next, I tried drive to drive tests (on the now XP Pro server) and found that my speeds increased to 80-90MB/s. This makes me believe that my problem is somewhere in the drives using Ubuntu, or at least that Ubuntu could do better.

Would the speed of the OS drive matter? Currently it's a rather old laptop drive I pulled from a scrap laptop. Also, I will also use HDtach to test my laptop's hard drives but I think they should be good since they are fairly new.

Thanks

Just remember that when XP is used as a server you need to set the system to use a large system cache for the best performance. Please take a look at this (http://forums.smallnetbuilder.com/showthread.php?t=1802) thread for more information.

Generally the OS drive would only affect your performance if it is the one being copied to/from. Could you do a hdparm (or HDtach) test on your server's OS drive?

00Roush

[milo]

Just remember that when XP is used as a server you need to set the system to use a large system cache for the best performance. Please take a look at this (http://forums.smallnetbuilder.com/showthread.php?t=1802) thread for more information.

Generally the OS drive would only affect your performance if it is the one being copied to/from. Could you do a hdparm (or HDtach) test on your server's OS drive?

00Roush

Sorry for the confusion, I didn't intend to use XP as a server software but just wanted to use it to test of the performance of the server system. Seeing that the disk to disk transfer speeds went up but the network transfer speeds went down helps me believe it's a configuration issue instead of a hardware issue.

I have since reinstalled Ubuntu 9.04 on the server and checked the OS drive with hdparm. The OS drive is a lot slower the "Timing cached reads" are about 574 MB/sec and the "Timing buffered disk reads" are about 14 MB/sec. The first reinstall I used the same setup as before except I decided to remove the swap partition since I thought the drive was slow. This didn't change anything. So I reinstalled again, this time with no swap and using the EXT4 filesystem on the OS and storage hard drives. The EXT4 filesystem gave me quite a boost. My drive to drive transfers went up to 77 MB/s and my network transfers went up as well. Using Vista 64-bit to the NAS, I see 55-60MB/s and using XP Pro to the NAS, I see 47-54MB/s.

I don't know if I can get much more from this system but moving to EXT4 from EXT3 did give me a good boost. I have a few more things to try from the reading I have been doing and I'll post any other results I get.

Thanks

Milo

[00Roush]

Glad to see your getting better performance.

Have you to gotten a chance to test your laptop's hard drive? I wonder if the speeds you are seeing now are the max speed of this drive?

00Roush

[milo]

Oops, sorry I forgot to mention those results. I downloaded the free version of HDTune and got the results shown below. Both hard drives have slightly higher average speeds than my current transfers so this might be the bottleneck now. The right (40 degree C) drive is the OS drive when running XP on my system.

Thanks again,

Milo

http://i39.tinypic.com/6fsd1u.png http://i41.tinypic.com/23ocjb.png

[bookmage]

Interesting stuff... I've been pondering using an Atom 330 for a mini-NAS for a while and this is helpful stuff...

On a slightly related note and cuz I like to crosscheck benchmarks w/lots of tools...
When I started using Linux, I was told to use Bonnie++ (http://www.coker.com.au/bonnie++/) to check disk speeds. This recent article helps describe how: http://www.linux.com/archive/feature/139742

On the windows side, I used DiskBench (http://nodesoft.com/DiskBench/Default.aspx) which doesn't have a lot of options, but it was good for giving me real life numbers and pinpoint network or local bottlenecks.

I'll have to add your tools and configurations to my testing process...

[00Roush]

Interesting stuff... I've been pondering using an Atom 330 for a mini-NAS for a while and this is helpful stuff...

On a slightly related note and cuz I like to crosscheck benchmarks w/lots of tools...
When I started using Linux, I was told to use Bonnie++ (http://www.coker.com.au/bonnie++/) to check disk speeds. This recent article helps describe how: http://www.linux.com/archive/feature/139742

On the windows side, I used DiskBench (http://nodesoft.com/DiskBench/Default.aspx) which doesn't have a lot of options, but it was good for giving me real life numbers and pinpoint network or local bottlenecks.

I'll have to add your tools and configurations to my testing process...

That is an excellent point.

Testing with HDtune or Hdparm basically gives you the raw disk speed and usually gives an idea of the maximum possible speed. The thing is that this is not the true real world speeds you will see. Most of the time actual read/write speeds are close to the raw speeds but it really depends on the file system being used. So using a program like Bonnie++ or DiskBench makes a little more sense. Iozone could also be used for this type of testing.

00Roush

[DarthV]

Has anyone used one of the dual core atoms to run a NAS? I'm looking at going with the dual core to get gigabit ethernet and slapping in a cheap 4 port SATA controller along with it. And probably going with a mimimal install of jaunty.

[aknab666]

Has anyone used one of the dual core atoms to run a NAS? I'm looking at going with the dual core to get gigabit ethernet and slapping in a cheap 4 port SATA controller along with it. And probably going with a mimimal install of jaunty.

Yes I have and it found it impressive given the power it uses. Unfortunately it took a power hit and is offline for the future.

[milo]

Has anyone used one of the dual core atoms to run a NAS? I'm looking at going with the dual core to get gigabit ethernet and slapping in a cheap 4 port SATA controller along with it. And probably going with a mimimal install of jaunty.

My setup listed in post #1 of this thread is working great as a NAS and FTP server. I still haven't gotten a Media Server installed on it, but that's only because of my lack of time and my lack of knowledge with Linux (Ubuntu).

I am just using the two on board SATA ports so I can't comment on a controller card.

[DarthV]

I'm planning to use the box as a media dump. So samba to my windows PC/laptop and NFS to my asrock ION 330 nettop running XBMC for my 52" LCD TV. Nice to see that you've got 50-60MB/s working with the onboard gigabit NIC.

[DarthV]

I've changed my mind, a friend wants to upgrade his HTPC, so I'm going to buy his current asus P5N7A-VM for a good price then throw in an intel e1400 + 4 port sata controller. Close to the same price as getting the dual core atom combo, but with extra PCI and PCI-e slots.