One of the truly noteworthy threads on the board details how to duplicate the contents of one DVR onto another:
However, if you have followed some of the threads here and on the DVR board over the last ten weeks, you may remember that I found myself in a situation in which my drive had suffered some unknown file corruption which prevented it from booting the DVR, and the process described in that thread produced a duplicate drive that inherited the same unbootable behavior. Nonetheless, in mounting my drive in Linux, I found that the file system was intact, and the recordings appeared to be present and accounted for. Surely, I thought, there ought to be a way to move them to a fresh drive.
This led to a long discussion that started here:
and later migrated here:
With lots of experimentation, trial, and error, I discovered how to transfer *individual recordings* from one drive to another. This could be useful for anyone with a little patience and who isn't afraid of issuing some simple unix commands. If, for instance, all of your recordings of one show are on one drive, and somehow you end up out of town and recording the latest episode to a different drive. Or maybe you want to archive some set of recordings to another drive. This process would work for that as well. You could even use it as a backup process, although the speed of transfer may work against that, vis-à-vis simply copying an entire drive.
Key point - this process will allow recordings to be transferred to a drive that already contains recordings without disturbing the existing recordings. (The same is true of the process for copying the entire drive.) That is critical.
Caveat - this does not defeat the restriction that recordings can only be played on the DVR on which they are originally recorded. Nor is it intended to do so - if you want to use this process to experiment with that, don't mention my name.
Caveat 2 - This requires being able to mount both the source and the target drive in Linux, which means an internal drive must be removed from your DVR. This again returns us to the issue of owned versus leased DVRs, which I'm not going to address - we all know that debate. In my case, the problem child drive was an external, so the issue didn't come up.
Caveat 3 - I have not figured out how to, or if it is even possible to, transfer scheduled recordings or series recording links between machines. Just recordings that have already been made. Maybe someone cleverer than I can do some research into that one. (P Smith, you game?)
ETA: Caveat 4 - This process seems to fail if the target drive is a Fantom Green Drive. This may be related to that drives non-use of the new Advanced format technology, but that's just a guess at this time. See post #10 for a longer discussion.
So, enough blabbering. How do you do this?
Stage 1 - Connect both drives to a Unix/Linux system and mount the filesystems.
This part is identical to the description for duplicating an entire drive, although I will name my drives differently. (The naming is not crucial, just helpful to me.) It assumes that both drives already have a DirecTV system on them. I was able to make this work even though the system on my source drive was slightly corrupted. Obviously, any corruption has to be minor enough that the file system itself and the recordings and their metadata are still intact.
It also assumes that both drives were "gracefully shut down" when last booted in your DVR. Otherwise, you'll have to perform an xfs_repair before they'll mount in Linux. (Search the forum for the invocation syntax of xfs_repair that works in this context.)
The drives can be connected via eSATA or USB - although the latter is about half as fast, and I experienced some reliability issues when copying files. I have used GPartEd Live 0.4.6-1, which works just fine. Although other versions of Linux are known to not work for the process of copying an entire drive, I’m tempted to see if my process will work in Ubuntu or some richer version than GPartEd.
Once you boot, do the following from the Linux command prompt to mount the file systems:
mkdir /mnt/new mkdir /mnt/old mount -t xfs -o rtdev=/dev/sda3 /dev/sda2 /mnt/new mount -t xfs -o rtdev=/dev/sdb3 /dev/sdb2 /mnt/old
Your device designations may be something other than "sda" and "sdb" - GPartEd will show you which is which. And be sure that you have the "old" and "new" designations right - if you have them reversed, you'll end up transfering recordings in the wrong direction.
Once you have things mounted, do the following:
cd /mnt/old/viewer/indexfile ls -1 (last character is a "one", not the letter between k and m in the alphabet)
You will see a full list of your recordings. If it's a long list, you may wish to pipe the output through the "more" filter. (A basic familiarity with Unix will be helpful, but you can get by without it.) Thanks to the DTV engineers, the directory names you see in this list give you a lot of information to help you identify your recordings. For instance, a typical name is:
The prefix is always the same. The next part is clearly the date, followed by the start time (using a 24 hour clock) in the form hhmm-ss. The next digit I don't know what it's for. Usually it’s a small number, but a few times I’ve seen it up in the hundreds. The next part is the channel number, and then the rest is always the same. Everything up through the "src2" I will refer to as the "program identifier", since the ".mpg" actually changes in one place.
At this point, to save myself a lot of typing, I'll use "xyzzy" to represent the program identifier, even though the actual names are much longer.
Now to transfer the recording, there are three bits of data that need to be transferred. Two are metadata of some sort, and the third is the actual recorded content. If you look closely at the folder /mnt/old/viewer/indexfile, you will see that the file "xyzzy.mpg" is not actually a file at all, but a directory that contains four files of metadata.
Similarly, the file /mnt/old/viewer/segments/xyzzy.mpg is also a directory - a very *large* directory that contains a series of 16 Mb files. These files are the actual recorded content, and they are stored on the real-time section of the file system. (As far as I can tell, no other data is stored on that partition.)
And finally, another bit of metadata lives in /mnt/old/mw_data/xyzzy.umd.dmd. That file stores the information on where you last left off when playing it, and I believe it stores the location of any bookmarks you have created in the recording.
So we need to move these three chunks of data to the target system, which lives in /mnt/new. The only catch to this is that the data in the "segments" directory has to be copied using the command "xfs_rtcp", since a regular invocation of the Unix command "cp" will copy it to the non-realtime portion of the file system. This portion is "only" 16 Gb, and it would fill up very quickly.
So, the commands to make the transfer are these:
cd /mnt cp old/mw_data/xyzzy.umd.dmd new/mw_data cp -R old/viewer/indexfile/xyzzy-min65535-src2.mpg new/viewer/indexfile mkdir new/viewer/segments/xyzzy-min65535-src2.mpg chmod a+rwx new/viewer/segments/xyzzy-min65535-src2.mpg xfs_rtcp old/viewer/segments/xyzzy-min65535-src2.mpg/* new/viewer/segments/xyzzy-min65535-src2.mpg chmod a+x new/viewer/segments/xyzzy-min65535-src2.mpg
One thing I discovered by accident is that if you do all but the last two steps, you’ve transferred all the metadata, as well as created the folder where the recording data will live. This can be done very quickly, since it’s only the xfs_rtcp command tat takes a long time. AS a result, if you boot the disk on your DVR, these recordings will appear in your list, although, obviously, you won’t be able to play them. This was useful to me because about a quarter of the recordings on my drive I couldn’t know what they were just from their file names. (The others, base on the date, time, and channel of the recording, I was able to figure out.)
You can also see that there is a lot of repetitive typing here. I've found that a moderate knowledge of unix commands is very helpful. The two things that were really useful were the auto-complete command, that will allow you to complete a path and file name by typing just a few characters at a time, as well as the history function, that allows you on the command line to call up previous commands and edit them to invoke new commands.
Also, because I had to copy 163 different recordings, creating a simple Unix script made life a lot easier.
This process is not something that a lot of users are going to need. But I'd never come across someone figuring out how to do this, and so now it's out there for anyone to use. If anyone tries it and discovered a typo in my commands as entered here, let me know. Unless you type something really wrong, all that's likely to happen is that Unix will complain that it can't find a file or a directory. Unless you have a recording on the destination drive with the exact same name (which I don't think is possible), I don't think this will break anything if you have a slight typing error.
Finally, be aware that the real-time copy command is, ironically, very slow. With my eSATA mount, it transferred about 5.5 Gb per hour. That means that a one-hour program in 1080i transfers in about 40-50 minutes. Recordings in 720p, or in standard def, will go correspondingly faster. It was also interesting that recordings of the same length and in the same resolution can take significantly different amounts of disk space - almost a factor of two, at the extremes.
Edited by cigar95, 19 September 2011 - 11:51 AM.
Discovered now information that may result in problems