The FTP Process
In a nutshell, FTP (File Transfer Protocol) provides an intermediate storage location (FTP site) for files on the web so that someone else can get them later.
The FTP process looks like this...
- We generate digital audio files for you by digitizing your tapes or records
- We login to the The Audio Archive FTP site with a username and password that only we know
- We put (upload) your digital files on the FTP site and place them in a password protected directory
- We provide you with a username and password that only you will use
- You login to the The Audio Archive FTP site with your unique username and password
- You get (download) the digital file from the FTP site and receive them on your computer
- Once you tell us that you have successfully received the files, we delete them from the FTP site.
You can also send digital audio files to us via FTP using the same secure FTP process.
We use MD5 checksums to guarantee that the exact same file that leaves The Audio Archive is also the same file you receive.
Often archival files (24-bit 96 kHz) are too large for FTP. A three-minute song (100 MB) is fine, but several one-hour concerts (2 GB each) would be impractical. In fact it would take over 18 hours to upload just one 2 GB file at 250 kbps - you can calculate this time using the FTP Transfer Time Calculator above!
We primarily use FTP for audio files under 200 MB each. For example, a 1-hour concert compressed in MP3 format (256 kbps CBR) would be about 110 MB. This will allow you to listen to the results quickly. If you are doing cataloging or transcription work, the catalogers or transcribers can begin listening to audio material much sooner.
FTP Bandwidth Converter
This converts bandwidth values commonly given in bits-per-sec (bps) to meaningful transfer rates, such as Megabytes-per-hour (MB/hour) or Megabytes-per-minute (MB/min). This will give you a quick sense of how fast or slow your internet connection is for the purpose of downloading or uploading files over the internet.
Typical bandwidth examples (bits-per-second)
- Dial-up modem = 64 kbps (enter 64,000 bits-per-second) download and upload
- ADSL = 1500 kbps (enter 1,500,000 bits-per-second) download
- ADSL = 300 kbps (enter 300,000 bits-per-second) upload
- T1 = 1.5 Mbps (enter 1,500,000 bits-per-second) download and upload
- Cable Modem = 6.0 mbps (enter 6,000,000 bits-per-second) download and upload
- T2 = 6.312 mbps (enter 6,312,000 bits-per-second) download and upload
- T3 = 45 mbps (enter 45,000,000 bits-per-second) download and upload
- T4 = 274 mbps (enter 274,000,000 bits-per-second) download and upload
The above bandwidth rates are theoretical maximums. The quality of your connection, TCP/IP overhead, the distance from your location to the central office, and the number of users at your location will all reduce your actual speed. Read more about internet speed here.
FTP Transfer Time Calculator
The Audio Archive uses this calculator to compute how long it will take us to send digital files to you via FTP. Our average upload speed (from us to the FTP site) for large files is about 250 kbps and our download speed (from the FTP site to us) is about 900 kbps.
Once you know your connection speed (upload and download - which will be different from ours), you can compute how long it will take to send a file to and from your computer to the FTP site. Simply enter your internet connection speed in the Bandwidth Converter, and then enter the size of your file (in Megabytes) and the number of files if all of the files are about the same size.
Internet Connection Speed
So just how fast is your connection? There are two speeds that matter:
- Download speed - how fast you can move a file from the internet to your computer, and
- Upload speed - how fast you can move a file from your computer to the internet.
How can you measure the actual speed of your connection?
One easy way is to use any of the free services available on the web, like www.speedtest.net or www.wugnet.com. These services copy a small file to your computer and measure how long it takes to do this (download speed), then they copy the same file from your computer back to the internet and again measure the time it takes (upload speed).
How accurate are the speeds computed by tools like Speedtest.net or WUGNET?
These speeds are good for small files (burst speed), but are not always accurate for large files (sustained speed). Often, the speed for large files is quite a bit slower than for small files (as much as 30-50% slower).
Sometimes the speed will vary depending on the time of day the test was performed. For example, cable modems share the same line with multiple users. If there are more users on the internet, your connection speed will be slower. If you are in an office environment, the number of people using the internet at any time will affect your connection speed.
Why are my download and upload speeds different? Why is upload so much slower?
Some connections, like ADSL, are biased towards faster download. In fact, the "A" in ADSL stands for asymmetric (which means that the upload and download speeds are not the same). ISPs (Internet Service Providers) provide different download and upload performance because most users are frequently receiving large amounts of information from the web to their computers (for example, music, movies, or YouTube videos), but are not sending nearly as much from their computers to the web. In order to provide affordable internet service, the ISPs make choices on where to cut costs - they do this by reducing the less frequently used upload speed (ADSL), sharing the same connection among multiple users (cable modem), load leveling (distributing the bandwidth among customers based on total demand), or any variety of strategies.
FTP Security
When individual logins, passwords, and file folders are used on an FTP site, this provides a high level of security. This means that each customer has a private location to put their data without any risk of other clients accessing their data. This is much different from the commonly used "anonymous FTP" where anyone can access a common directory without a login or password. The Audio Archive provides secure password-protected FTP services at no charge.
FTP Software
There are many FTP client software packages available. A good FTP client does a number of things:
- Manages your FTP logins and passwords
- Provides an easy-to-use GUI for handling large batches of files
- Allows you to easily browse your local file system or the remote FTP site
- Allows you to monitor the upload and download status for each file
- Automatically reconnects if the FTP connection is lost
- Automatically continues the FTP process where it left off instead of starting over if the FTP connection is lost (if this feature is available on the FTP server).
- Takes advantage of multiple connections to a single FTP site to maximize file transfer rates
- Rock solid reliability so that you can leave your FTP transfers running in the background and overnight
Our favorite tool for FTP is the GlobalSCAPE CuteFTP Professional software. It has paid for itself many times over in time saved and files delivered on time. We could not live without this software. GlobalSCAPE has been around for many years, and they provide excellent support for their products. Unfortunately, the CuteFTP software is only available for Microsoft Windows.
FTP Accuracy and Checksums
In order to ensure that the file we send is exactly the same file you receive, we send you a digital signature for that file, called a "checksum" or "hash". There is a unique checksum for every file. If the checksum of the file you receive matches the checksum value that we sent to you, then there were no errors and the file you received is a perfect copy of the file we sent. The checksum works for any digital file, not just digital audio, but text files, binary files and other media, too.
The checksum process looks like this:
- We compute a unique MD5 checksum (digital signature) for each audio file
- We put each MD5 checksum in a file with the same name as the audio file, but with a different extension (.md5 extension)
- We send both the audio file and the MD5 checksum file to you (via FTP, hard drive, CD-R, DVD-R, or LTO)
- You compute checksums for each audio file you receive, and see if they match the checksum values that we included.
In fact, the use of checksums for each file is an essential element of preservation. As digital files are moved from one location to the next, or moved to new media, it is critical to make sure that the files have not changed.
Even if the digital files remain in the same location on the same media, it is important to check if the media is degrading in any way over time - and this can be done, again, by checking the checksums to ensure file integrity. If any files are corrupted, you will know exactly which ones.
If the files are the same size, aren't they the same?
Unfortunately, no. If the files are not the same size, certainly something has changed. But if the files are the same size, that is no guarantee that they are identical. For example, you can edit a text file and change one character without changing the number of characters. The edited file will be the same size as the original, but the content of the file has changed. The MD5 checksum can catch these changes - even if only one bit changes.
MD5 Checksum Software
Without the right software, computing and comparing checksums for thousands of files can become very difficult and time consuming. Fortunately, we have tested a wide variety of freeware, shareware, and good old-fashioned software. After reviewing many software packages, we chose FastSum as the most capable standalone software for generating and comparing checksums. Unfortunately, FastSum is only available for Microsoft Windows.