CHIP IT

I try to find some candidate hardware platforms and Linux distributions in this article, so the fans can build their own Network Attached Storage (NAS) or expand their NAS with more features by hacking an existing NAS (Network Attached Storage).

What is NAS?

NAS is the abbreviation of Network-attached Storage. It was introduced by Novel to offer the network file sharing service. Network attached Storage (NAS) was designed for enterprise applications, so it supports UNIX from the very beginning. The consumers realized they need more storage capability for the digital media files from Internet. A consumer NAS (Network-attached Storage) can share the media files with all the PCs and the digital media players. Furthermore, NAS can download the files from BitTorrent or eDonkey in a non-PC environment. The latest consumer NAS transforms into a full-functional media server with variety features.

Let us check out the long list:

1. File sharing for Windows, Linux and Mac via Samba, NFS, HTTP, FTP and rsync;
2. Easy data backup capability for flash cards, USB stick and removable HDD, with optional RAID support;
3. A print server;
4. A Media server for Windows MCE, Xbox360 and PS3 with UPnP/DLNA;
5. A Web server with DDNS, PHP, ASP, SQLite and MySQL;
6. An FTP server;
7. An iTunes server;
8. A 24 hour download server supports BT, eDonkey and FTP;
9. A home video surveillance server, which supports both IP camera and USB camera;
10. Multiple administration panel choices in Web GUI, virtual console and custom terminal software;
11. Much more …

Why Do We Need NAS (Network Attached Storage)

Although we can use the more powerful PCs in many applications, we still require embedded computers as NAS for security, power consumption, and network.

Security

A well-designed NAS can offer more security than a regular PC. Most of the NAS OSes are based upon Linux OS, so they have less virus problems compare to Windows. Even if they have downloaded the files infected by some viruses, the embedded OS in the NAS will not be infected by the viruses. Of course, you still have to scan the files with the anti-virus software.

A dual slot NAS usually offers RAID backup service. So the important data can be restored if one disk has a malfunction.

Power Consumption

A PC is not designed for working 7*24 hour. It can work for a long time. But I will not leave my PC to download a huge file without knowing when it can be ended. A NAS can work in more energy effective ways. If there is no active connection, the NAS can shut down the hard disk drivers and wait for the connections. The power consumption of a regular NAS depends on its hard disk drivers, which is about 5W~20W. A NAS can save our money and environment with less carbon emission.

Network Access

A Network-Attached Storage (NAS) can reduce unnecessary network communication to Internet. Most of the media files are shared on the NAS within the LAN. The other IP based appliances can work with the NAS by fetching the digital media files to playback. We can setup our own “HOME” page at home, because a NAS also offers a web server with DDNS (Dynamic DNS). Besides the regular web content, we can use it to access to the cameras as a remote surveillance server. It is a very important service for digital home. A NAS is a good platform to host such service.

Actually the NAS could be a platform for many commercial network applications. For example, you can use it in a Bluetooth advertisement pusher or a music retail store. Just plug in a Bluetooth stick and install proper software, the NAS can start to push the advertisement or the sample music clips via Bluetooth.

Paradox of NAS

There is a paradox for NAS. A NAS supposes to offer much larger storage capacity than the existing storage media. However, due to the fast growing of storage market, the storage capacity of a NAS usually seems smaller after several months. According to the famous Moore's Law, a NAS usually faces to the upgrading requirement for disk capacity and local network connectivity bandwidth in every 18 months. Today, an advanced desktop PC has a 512GB hard disk and a NAS has a 2TB hard disk. Maybe a regular PC will have a 2TB hard disk 6 month later. So the NAS will lose its capacity advantage at that moment. A consumer may hesitate to offer a NAS which seems to be obsolete in one year anyway. As a result, the consumer class NAS integrates many entertainment features, so the consumers can convince themselves that NAS can help in many other ways besides storage and sharing.

NAS Hardware Platform

The latest NAS model features SATA-II slots, USB 2.0 high speed host ports, Gigabit Ethernet or 54M WiFi connectivity. The structure of a NAS is identical to a regular PC without VGA and input devices. In general, either a RISC based embedded application computer or a regular x86 PC can be used as the hardware platform for a NAS. Most of the commercial NAS devices use the RISC processors. It is your own decision to pick the hardware platform. However, developing on a RISC embedded system requires extra hardware tools and knowledge, which is not recommended for a beginner. Building a PC based NAS is a good starting point for a beginner. At least we will not worry about breaking the board.

RISC based

I browse the data catalogue of the semiconductors manufacturers for NAS. Among these chips, Marvell’s Digital Home Platforms integrates most of the desired features with a 1.5GHz ARM chip, SPDIF/I2S audio and TS/Video port. Obviously, it is more than a NAS. The chip is very popular in the commercial NAS products. You can easily find one and hack it.

If you are looking for other platforms, you can also find the solutions from Freescale (Power PC based), PMC-Sierra (MIPS based), and Cirrus Logic (ARM920 based). However, these chips usually have slower microprocessors (from 200MHz to 400MHz) and slower peripherals (ATA, USB1.1, 10/100M Ethernet). So you can only use it as a pure storage device.

If you want to build it from scratch to learn how Linux works, the best candidates are S3C2410/2440 and XScale. These chips are not designed for the NAS, but for hand-held devices. However, these chips are very popular and they have the most of the peripherals on chip. Of course, the performance of the peripherals are not enough, sometimes you have to expand it with supplementary hardware.

JTAG and Serial Console

You may need a JTAG board and a serial port for bringing up a 'virgin' board. You must use the JTAG board to download the boot-loader to the on board flash memory, and use the serial port to give the boot-loader the further programming and configuration instructions. However, if the chip (such as TI OMAP3530) has build-in boot-code to support extra booting options, such USB stick, SD card, Ethernet, hard disk driver, then you are lucky to select an easy chip to work on.

Microcontroller

Some NAS devices use a low cost microcontroller to get the user input and light the status LEDs. There are some hacks available for these microcontrollers. However, you have to get some hardware programming tools to download the microcontroller’s firmware. The programming method is vendor dependent.

PC based

As I mentioned, we can use an obsolete PC as the hardware platform, since the hardware infrastructure of a NAS is almost identical to a PC. We only care about the power consumption, scalability and availability for a specific hardware platform. An obsolete usually consumes a lot of power, generates a lot of noise. I don't like that reuse idea. A PC based NAS must be low power, low noise and fast enough.

If we check the x86 suppliers, Intel, VIA, AMD and their partners have offered many reference designs. Most of the reference designs are based upon low-power CPUs. Among these chips, I prefer the ATOM processor from Intel. It offers the lowest power consumption so far. The ATOM based main board is about 100USD. You can setup a dual bay NAS on ATOM platform within 140USD (excluding SATA HDD). The EPIA board from VIA is also a good candidate for x86 based NAS.

You can leverage the PC to develop your NAS with minimum cost. The cheap peripherals allow you to expand your NAS in many ways, such as WiFi connection, flash card bay, IEEE1394, eSATA and Bluetooth. You can download and install many open NAS OSes to find out the suitable package. Besides, you can install the OS on CD-ROM, USB stick or HDD without programming flash in an embedded system. Further more, you can expand your NAS to a media server with extra software modules.

NAS Software Component

We should always look for the software components from open source communities, because most of the NAS software packages are Linux based distributions (although Microsoft SAK [Server Appliance Kit] and VxWork also have a small market share). Most of the semiconductors suppliers offer reference design with Linux BSP as well. Some vendors like Cirrus Logic offers full source on its site, the other vendors only offer their packages for their clients. So you have to find the alternative Linux distribution from the open source community sites if you can not get the packages from the semiconductors suppliers. Fortunately, there are too many options for you.

First, there are off-the-shelf distribution from Debian and famous Linux distributors. You can install this distribution easily.

Second, you can build the Linux with open data sheet and open source tools by yourself. The other applications in user land can be build later and then install to the NAS.

Complete NAS OS

FreeNAS supports CIFS (Samba), FTP, NFS, rsync, AFP protocols, iSCSI, S.M.A.R.T., local user authentication, and software RAID (0,1,5), with a web-based configuration interface. FreeNAS is a Live CD distribution of FreeBSD, which can be installed on a 32 MB CompactFlash, hard drive or USB flash drive.

Openfiler was created by Xinit Systems, and is based on the rPath Linux distribution. Openfiler needs at least a 500 MHz CPU, 256 MB of RAM, 1.2 GB hard disk space, an optical drive and a FastEthernet network interface.

NASLite is a Linux distribution designed to turn conventional x86-based computers with PCI interface into a simple network-attached storage device. NASLite boots from the floppy disk and runs in a 4MB RAM disk allowing for full capacity of the hard disk drives to be used as storage. Now NASLite becomes proprietary software, without source code.

The CryptoNAS is another live-CD project to offer encryption of user data in a NAS.

The above NAS OS distributions can not offer every feature you want, then you can custom and strip down the Debian and FreeBSD to meet your requirement.

File Sharing

The basic file sharing services include NFS (for Linux), Samba (for Windows). Both services have been implemented already. In fact, any Linux computer can work as NAS, because file sharing is a default service for Linux. The only concern is file system for the hard disk. Usually the internal hard disks are formatted as EXT2/EXT3/UFS. That is transparent for the client PCs. But the developer must hack the kernel to support NTFS, especially in writing a USB hard disk formatted in NTFS.

Data Backup

NAS normally uses RAID as back up feature. There are various combinations of these approaches giving different trade offs of protection against data loss, capacity, and speed. RAID levels 0, 1, and 5 are the most commonly found, and cover most requirements. Actually RAID level 0 just increases the capacity and speed, no any data backup is available. And RAID level 1 is a mirror backup, so you can restore the data in case one disk fails, but keep in mind that even if two HD of the same size are used, the total RAID volume capacity equals the capacity of one HD. The RAID level 5 offers high security for data restore. The RAID can be implemented in software, hardware and firmware. For Linux, it is supported in the kernel already. Even one disk failed in software RAID1 system, you can still read out the other hard disk on other hardware. If you configure the RAID system in other higher level, you need to restore the data in the same device.

Print server

The printer sharing is implemented in SMB of Samba package. In order to install the Linux network printer for Windows workstation, you have to:

1. Install and configure the Samba server
2. Add extra script in /etc/printcap
3. Create a filter file under /var/spool/lpd/smb/
4. Create a .config file under /var/spool/lpd/smb/
5. Restart the printer by #lpc restart all
6. Find the network printer in Samba from Windows workstation
7. Install the printer postscript driver in Windows
8. Print from Windows workstation

Please refer to Linux printer how-to for more informations.

P2P Sharing

The P2P sharing must be included in a consumer NAS. QNAP, ASUS and D-Link have integrated BitTorrent already. The BitTorrent client has many Linux ports, which can be cross-compiled in an embedded system. Another well-known P2P sharing protocol is eDonkey. eDonkey is open source software as well. But the first source is released for Windows VC++. Compared to BitTorrent, eDonkey has less Linux ports. So far, the eDonkey download feature is only included in some high-end consumer NAS. According to some reports, the download speed of the embedded P2P clients is much slower than the PC based clients. So far I didn't find the official analysis report.

There is a complete and update list for BitTorrent clients in Wikipedia, which are sorted by UI, programming languages, OS and etc. We need a web based, C++ based client software. Of course you can port a Java or Python client if your system supports these languages.

The consumers want to have these two major P2P protocols in one package. Some commercial NAS devices use a combination of Clutch+Transmission. The other ones use MLDonkey, which supports BitTorrent, eDonkey and other protocols. This software is written in Objective Caml, C and assembly. Porting Objective Caml is the major task for MLDonkey.

The user front-end usually is based upon web interface. If you are interested, you can use the custom software. Sancho is a good front-end software working with MLDonkey. You can access the P2P service in a Windows workstation just like running the eMule software in the native PC.

UPnP/DLNA media server

The UPnP architecture allows local peer-to-peer networking of PCs, networked appliances, and wireless devices. It is a distributed, open architecture based on established standards such as TCP/IP, UDP, HTTP and XML. The UPnP is a media independent protocol.

The Digital Living Network Alliance is an international, cross-industry collaboration of consumer electronics, computing industry and mobile device companies.

The DLNA is an industrial standard, so more and more media players integrate UPnP/DLNA support, such as PowerDVD, WinDVD, Vista, PS3, VLC media player and famous MPlayer/GeeXbox.

In order to support UPnP/DLNA media server in NAS, you have to find libdlna, ushare for Linux.

If you are going to setup your home entertainment network, you definitely need one or several NAS servers to share the media files. Switch on your front-end equipment, IPTV, PS3, Xbox, DMA, HTPC, enjoy yourself!

Web Server

Some RISC based commercial NAS claimed they are offering Apache+MySQL web server inside the NAS server. I am a little confused because I know a full-featured Apache+MySQL+PHP combination is hard to be cross compiled. Normally, the web server for an embedded system will use small footprint server like BOA. The LAMP combination is working perfectly on a PC based NAS.

FTP Server

The FTP server and client have been integrated into Linux long time ago.

IP Camera

The USB camera is very cheap and widely used. Some of the NAS devices can use these low-cost USB cameras as video surveillance sensors and embedded the real-time video/audio stream in a web page, which can be accessed by authorized administrator members via DDNS. Porting USB camera requires some knowledge about the interface IC. You must identify the semiconductors suppliers and find the corresponding drivers. Please visit Video4Linux and Linux USB for the latest information about camera.

The IP camera is a better choice for video surveillance. It usually offers better video quality (SD/HD on higher frame rate), long range operation (Ethernet or WiFi), PTZ (Pan, Tilt and Zoom) controls, audio support, advanced video format (MJPEG, MPEG-4), night vision, PIR input and more. It is easy to integrate an IP camera with a NAS, because it is an 'IP' appliance.

You can even merge video inputs from several cameras in one web page to monitor multiple sites in the same time. It requires web programming skills.

Hacking a Commercial NAS

Hacking a commercial NAS is a good approach to learn the embedded Linux. There are too many resource web sites for hacking a commercial NAS. You can modify the firmware and add extra features for your own purposes. Although a commercial NAS has limited scalability, you still can enjoy yourself in hacking such a high-tech toy. Why not? Just buy a second-hand NAS from ebay and hack it.

You can visit NAS-Central.org as the starting point. This wiki site will lead you to the specific brand, model and hacking blogs. You can follow it to know what has been done by other developers, what you can do and what you can not do.

A quick search on Google reveals that there are a lot of free serial port monitor applications available for PC users. However, what these applications lack, is the possibility of controlling the serial port in a direct manner. They are generally good “sniffers” but they do not allow the user to actually write to the serial port or control any device attached to it. The applications with the write capability encapsulated are not for free, and the cheapest costs about 50 Euro – a great deal of money taking into account how easy it is to make a personalized application.

This article will show how it is possible to build such an application using the C# environment. It is not intended to be a C# tutorial, but to teach a user who has basic knowledge of C or C# to integrate serial port control in one of his applications.

For the example application, I have used the SharpDevelop development environment which includes a C# compiler. This is an open source IDE which takes up very little space on your hard drive and can be a good alternative to users who do not want to install the gigabytes of Visual Studio on their PCs for a simple serial port application.

Once you have downloaded and installed the SharpDevelop environment, create a Windows Application project (solution) called SerialPort:

Once you have created the application, display the windows form that was automatically created (by clicking on the “Design” button at the bottom of the screen) and unroll the menu available under “Components” available on the left-hand menu:

You will notice that one of the components available here is the one called “SerialPort”. Pick that component and drag&drop it over the surface of the form on the right. This will add the component to your project. The object that is created is called “serialPort1” and it will be used to access the serial port. To be able to use this component, however, you need to add at the beginning of your code the directive for using the System.IO.Ports namespace, as this is not added by default when you create the solution:

using System;
using System.Collections.Generic;
using System.Drawing;
using System.Windows.Forms;
using System.IO.Ports;

Once this is done, create a regular button on the surface of the form, call it “button1” and change its label to “Write”. Then double click on it in order to create the function that will be executed when the button is clicked.

In this function we will perform several tasks. The first one is to configure the baud rate, COM port, number of data bits, parity and stop bits of the communication:

//configuring the serial port
serialPort1.PortName="COM1";
serialPort1.BaudRate=9600;
serialPort1.DataBits=8;
serialPort1.Parity=Parity.None;
serialPort1.StopBits= StopBits.One;

Next, before writing to the port, it needs to be opened:

//opening the serial port
serialPort1.Open();

Please note that if the COM1 port is already used by an application, you will get an error message when this instruction is executed. Alternatively, if you open the COM1 port with your C# application and then fail to close it, any other application trying to use it will not be able to do that.

OK, it is now time to write to the serial port:

//write data to serial port
serialPort1.Write("ABC");

When this instruction is executed, three bytes are sent to the serial port: the ASCII code of “A”, the ASCII code of “B” and the ASCII code of “C”.

Once the write operation is performed, you must not forget to close the port:

//close the port
serialPort1.Close();

So, as a summary, all the code that makes up the body of the function should be:

void Button1Click(object sender, EventArgs e)
{
//configuring the serial port
serialPort1.PortName="COM1";
serialPort1.BaudRate=9600;
serialPort1.DataBits=8;
serialPort1.Parity=Parity.None;
serialPort1.StopBits= StopBits.One;

//opening the serial port
serialPort1.Open();

//write data to serial port
serialPort1.Write("ABC");

//close the port
serialPort1.Close();

}

Now you have a broad path open in front of you, as you will be able to write your own customized applications that can send any data to any device attached to the serial port. The above code is also compatible with USB to Serial converters, provided that their drivers work by emulating COM ports.