Jump to content


Welcome to DBSTalk


Sign In 

Create Account
Welcome to DBSTalk. Our community covers all aspects of video delivery solutions including: Direct Broadcast Satellite (DBS), Cable Television, and Internet Protocol Television (IPTV). We also have forums to discuss popular television programs, home theater equipment, and internet streaming service providers. Members of our community include experts who can help you solve technical problems, industry professionals, company representatives, and novices who are here to learn.

Like most online communities you must register to view or post in our community. Sign-up is a free and simple process that requires minimal information. Be a part of our community by signing in or creating an account. The Digital Bit Stream starts here!
  • Reply to existing topics or start a discussion of your own
  • Subscribe to topics and forums and get email updates
  • Send private personal messages (PM) to other forum members
  • Customize your profile page and make new friends
 
Guest Message by DevFuse

Photo
- - - - -

How to: Copy and Replace Internal Hard Drive


  • Please log in to reply
619 replies to this topic

Poll: I have read the disclaimer and I understand that this process in NOT recommended (598 member(s) have cast votes)

I have read the disclaimer and I understand that this process in NOT recommended

  1. YES - I agree I will not tamper with my leased equipment (414 votes [69.23%] - View)

    Percentage of vote: 69.23%

  2. NO - I agree to accept full financial responsibility for breaking my Customer Agreement (184 votes [30.77%] - View)

    Percentage of vote: 30.77%

Vote Guests cannot vote

#321 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,727 posts
  • LocationBay Area
Joined: Jul 25, 2002

Posted 02 May 2011 - 10:18 AM

If you have Ubuntu 9.04 you could upgrade its kernel to 2.6.39 by the procedure: http://www.ramoonus....d-debian-linux/

...Ads Help To Support This SIte...

#322 OFFLINE   ntrance

ntrance

    Legend

  • Registered
  • 145 posts
Joined: Aug 18, 2006

Posted 02 May 2011 - 04:40 PM

If you have Ubuntu 9.04 you could upgrade its kernel to 2.6.39 by the procedure: http://www.ramoonus....d-debian-linux/

Right. That's what I did, except the I used Ubuntu 11.04 as a starting point. I don't think the 2.6.39 kernel is available for older Ubuntu versions using that method. However, this is still not ideal as it requires installing Ubuntu, upgrading the kernel, installing xfsdump, and rebooting before doing the dump/restore rather than simply using a live CD. So I think it is best to keep using gparted-live-0.3.7-7 to gparted-live-0.4.8-6 until gparted or some other distribution release a live CD using the 2.6.39 kernel. My point was simply that the problem seems to be fixed and that we may not be stuck using old versions forever.

#323 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,727 posts
  • LocationBay Area
Joined: Jul 25, 2002

Posted 02 May 2011 - 05:04 PM

That's good to know the procedure still applicable to newest version of Ubuntu 11.04.
Will wait for new Gparted with a kernel v2.6.39.

#324 OFFLINE   CCarncross

CCarncross

    Hall Of Fame

  • Registered
  • 7,058 posts
  • LocationJackson
Joined: Jul 19, 2005

Posted 03 May 2011 - 06:12 AM

This seems like people are making a huge deal out of using a simple boot disk. THe beauty of the boot disk is, it doesnt change, and in this case, doesnt need to change to work as needed. When I did my 1st copy, I d/l'd the recommended distro, and still use the exact same old disk to this very day...this isnt like running a full time OS on your device, you dont need the latest greatest, as it makes absolutely no difference to how the process works.

#325 OFFLINE   ntrance

ntrance

    Legend

  • Registered
  • 145 posts
Joined: Aug 18, 2006

Posted 04 May 2011 - 01:54 AM

This seems like people are making a huge deal out of using a simple boot disk. THe beauty of the boot disk is, it doesnt change, and in this case, doesnt need to change to work as needed. When I did my 1st copy, I d/l'd the recommended distro, and still use the exact same old disk to this very day...this isnt like running a full time OS on your device, you dont need the latest greatest, as it makes absolutely no difference to how the process works.


The problem is that instructions say to use gparted-live-0.3.7-7 or later. This has lead to numerous people having problems, because they chose to use the latest version without first bothering to read the newer posts in the copy threads advising against that. I have already asked marty45714 to update the original post to be more explicit to hopefully avoid more of this in the future. I also posted a bug report for the xfs team a while ago about issues with the newer kernels and realtime subvolumes. It seems they have finally addressed the issue. While I agree with you that a single version of gparted is likely sufficient for the vast majority of cases, I think it is good if the root incompatibility is fixed. That way if some improvement is made (in speed for instance) or some new piece of hardware in someone's computer is incompatible with the older live CD there will be an easy path forward.

#326 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,727 posts
  • LocationBay Area
Joined: Jul 25, 2002

Posted 04 May 2011 - 10:09 AM

I would add to that more important (IMO) point - we are true followers, we're not in charge of modification of original kernel of the DRVs, we're not responsible for writing drivers of new kind of drives (eg 4 KB sector), we're accommodating small empirical knowledge - we're working with the DRV's drive as with a black box (perhaps I should say 'gray' :) ).
Thus, my pitch - we can't and shouldn't avoid new version of program/OS what we are using for our humble processes.
[Soon MPEG-2 will be ceased and those solid working receivers will be obsolete as old versions of Gparted.]

#327 OFFLINE   wnissen

wnissen

    New Member

  • Registered
  • 5 posts
Joined: Aug 03, 2008

Posted 09 May 2011 - 09:01 PM

Thanks for your help. I have a couple of comments. First, it is far easier to clone the disk if all you are looking to do is switch to the same size drive. That is what I was trying to do due to very flaky playback. This procedure is only relevant if you are switching to a larger drive, so it would be nice to note that. I also used "-p 60" instead of "-p 600" to get more timely status updates. Also, to boot to the CD on a Mac, hold down the "C" key while turning on the power. Finally, I can verify that this procedure works just fine with USB, which is what I used to connect the drives. Just plug in the drives one at a time and note what "/dev/sd" it is. It took about 16 hours to transfer 1.5 TB over USB, though.

#328 OFFLINE   mitchell357

mitchell357

    New Member

  • Registered
  • 2 posts
Joined: May 23, 2011

Posted 23 May 2011 - 01:12 PM

I used the -600 in the command line thinking that would give me periodic status updates. After entry of the command line, I got about half of a page of detail, ending in 'xfsrestore: restoring non-directory files'.

I launched the xfsdump... xfsrestore... command about 18 hours ago copying from a 500 GB to a 1 TB drive. Should I be concerned that no more status updates have been seen, and more importantly how much longer should I wait for the copy to complete?

Thanks - Mike

#329 OFFLINE   ntrance

ntrance

    Legend

  • Registered
  • 145 posts
Joined: Aug 18, 2006

Posted 23 May 2011 - 11:50 PM

Make sure you are using a known working version of gparted per the posts on the previous page.

#330 OFFLINE   mitchell357

mitchell357

    New Member

  • Registered
  • 2 posts
Joined: May 23, 2011

Posted 25 May 2011 - 08:00 AM

I quit from the xfsdump terminal session as it appeared to be locked up, no further status messages seen. In fact, I could not even get Gparted to do a graceful shutdown, I needed to force a hardware shutdown. I then booted up with the older version of Gparted ( 0.4.6-1 ) per recommendation of earlier posts, and received a message about needing to add a -R or -Q on the command line of xfsrestore in order to either restore or force completion. At this point, I thought it best to insert the internal drive back to the DVR, along with connecting the external drive, so I could do one more reset from the menu path.

Now with boot up under Gparted version 0.4.6-1 I still got the same error about needing the -R or -Q on the command line. Adding -R did not work, I needed to add the '-Q' parameter to the xfsrestore command line in order to force it to complete. And once I did this, it threw about 3 pages of warnings about 'bad file descriptor', so I thought I might be in trouble. But I let it continue to run, and it started kicking out status messages every 10 minutes about % complete, so it appeared to be working. When it got done, I put the DVR back together fired it up with the external drive and it does appear to be working! I now see 58% free space vs. the 5% free on the internal drive.

In summary, as earlier posts have indicated I found that it is crucial to use Gparted v. 0.4.6-1 after proper shut down of the DVR via the Menu - Reset path.

Thanks - Mike

#331 OFFLINE   cigar95

cigar95

    Legend

  • Registered
  • 135 posts
Joined: May 23, 2008

Posted 07 July 2011 - 11:11 AM

Admittedly, I'm prepared to be asked "are you out of your mind?"

I had an external drive that stopped booting a few weeks ago. (Discussed at length over on the DVR board.) Hardware diagnostics and SMART data indicated that there was nothing physically wrong with the drive, and UNIX diagnostics show that the file system is intact and readable - all the recordings are there, although of course you can't do anything with them in LINUX. My conclusion is that the *content* of some file had been corrupted, and that's why it wouldn't boot.

Having gotten a new drive, I figure I have nothing to lose, other than a half hour, by trying to copy the recordings over, using the instructions here. And here's where I'm getting nuts.

If I prepare a new drive with all the necessary operating system files, then all I would need to transfer would be the recordings themselves, and whatever catalog data might be necessary. (I'm willing to sacrifice my series links, there aren't that many of them. I'm willing to risk missing new episodes of 'Lost' and '24' if I forget to program them!)

So what if I don't copy ALL the files? since if I duplicate a file that was causing the problem, I'll just transfer the problem to the new drive.

In other words, do we know enough about the file structure that the DVR uses to know which files/directories must be copied, and which do not, given that we're copying to a working DTV system that already has the necessary files to boot and operate?

See, I told you I was out of my mind, but if anyone has knowledge of this, do tell! Many thanks, and be gentle.

Edited by cigar95, 07 July 2011 - 11:34 AM.


#332 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,727 posts
  • LocationBay Area
Joined: Jul 25, 2002

Posted 07 July 2011 - 12:08 PM

Nope, no knowledge - do copy as is.

#333 OFFLINE   CCarncross

CCarncross

    Hall Of Fame

  • Registered
  • 7,058 posts
  • LocationJackson
Joined: Jul 19, 2005

Posted 07 July 2011 - 01:40 PM

Currently, the only method involves copying entire drives...it does a bit for bit copy, there is really no method for copying one folder of shows, or individual shows, because there is no easy way to identify what files go to what physical show...so right now its all or nothing.

#334 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,727 posts
  • LocationBay Area
Joined: Jul 25, 2002

Posted 07 July 2011 - 02:25 PM

Actually it is DIFFERENT type of copy: a partition ie folders/files/metadata to a partition.

#335 OFFLINE   cigar95

cigar95

    Legend

  • Registered
  • 135 posts
Joined: May 23, 2008

Posted 07 July 2011 - 04:16 PM

Currently, the only method involves copying entire drives...it does a bit for bit copy, there is really no method for copying one folder of shows, or individual shows, because there is no easy way to identify what files go to what physical show...so right now its all or nothing.


Actually, if you look at the file structure, there's a folder called, I think, "viewer/segments", or something like that, which includes all the MPEG data, with a file/folder name that includes date, time, and channel, which clearly distinguishes individual recordings. This is the folder that includes hundreds of gigabytes.

That part seems fairly straightforward. There's another folder elsewhere that includes the same recording-specific names, but is far smaller. Maybe some sort of catalog data.

Then there's all the *other* stuff on the drive, whose function isn't clear from the various file and folder names. It isn't clear which of these need to be replicated on the new drive, and which ones can be ignored in favor of the versions already on the newly-created drive.

The other challenge in this regard is that it appears that xfsdump itself is strictly an all-or-nothing process - doesn't appear that you can give it file arguments to specify which files get copied, as one could do with a cp or mv command.

If one *really* wanted to try what I'm thinking of, the only option would be to *delete* everything that one doesn't want copied. In this case, the process is destructive - if you guess wrong on which files you need, it's too late. So you'd get *one* shot at it. (I suppose one could move all the unwanted stuff into a folder with a name like "oddball stuff" and hope the OS ignores it.)

Just for the sake of scientific knowledge, it might be interesting to create the virgin drive, make one recording, then mount the two drives in GPartEd, and use a standard Unix cp command to move what we think is all the associated data for *one* recording from the old drive to the new drive, and see if the system recognizes it after a reboot.

What are the chances of identifying all the files necessary to move a single specific recording? Probably pretty slim. If this could be accomplished with a cp command, someone would probably have figured it out by now.

But the fact that others have used the xfsdump process to move data to a drive with existing recordings and did NOT wipe out those recordings while moving ones from the other drive tells us something about the organization of the data. It seems to say that there isn't a single catalog file, but more likely a collection of files corresponding to individual recordings, such that the *folder* constitutes the catalog.

The more I type this out, the more I see how crazy it probably is. But crazier things have happened.

Edited by cigar95, 07 July 2011 - 05:15 PM.


#336 OFFLINE   ntrance

ntrance

    Legend

  • Registered
  • 145 posts
Joined: Aug 18, 2006

Posted 08 July 2011 - 04:40 AM

But the fact that others have used the xfsdump process to move data to a drive with existing recordings and did NOT wipe out those recordings while moving ones from the other drive tells us something about the organization of the data. It seems to say that there isn't a single catalog file, but more likely a collection of files corresponding to individual recordings, such that the *folder* constitutes the catalog.

Copying individual recording certainly seems possible, but I don't know if it is worth the effort to manually gather all the necessary files. It is easier just to dump a whole drive, and delete what you don't want from the new drive. You can even dump another drive onto that same new drive to consolidate recordings from the two old drives (provided the were attached to the same DVR) onto one new drive as you mentioned. I tried it here:
http://www.dbstalk.c...477#post2229477

#337 OFFLINE   cigar95

cigar95

    Legend

  • Registered
  • 135 posts
Joined: May 23, 2008

Posted 08 July 2011 - 10:16 AM

Thanks, ntrance, I have read the post a couple years ago about the two-drive experiment. The reason I was thinking about trying to copy just certain files is that I believe my existing drive has a corrupt file that prevents it from booting.

The purpose of my drive copy is as one last attempt to preserve the recordings, which appear to be intact. If I copy the whole disk, I'll likely copy the bad file and just transfer the problem to the new drive. Since I have no idea which file is bad, the idea was to try to copy as few files as possible and see if recordings can be transferred that way.

Ultimately I think the thing that might torpedo this idea is the use of the real-time partition. The partitions are mounted in such a way that they look like a single partition to the user. It isn't clear if a simple cp command will copy them to the proper partition, although a quick web search appears to indicate that a real-time file has the attribute set as part of its metadata. If that's the case, a cp command might automatically put it on the proper partition.

The other big question mark on all this is identifying all the files necessary for the system to recognize a particular recording. As mentioned above, I've identified two files that, based on their file names, have obvious associations to a given recording. but that doesn't mean there aren't other necessary files that can't be identified simply by their names.

The single biggest reason I think this is a longshot to work is that if it did, someone else on the forum would have already discovered it.

#338 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,727 posts
  • LocationBay Area
Joined: Jul 25, 2002

Posted 08 July 2011 - 11:36 AM

cigar95, I would answer to all your questions if I would do a few tests with empty eSATA drive:
- write one program, unmout safely and check under Linux what created there (perhaps post all three lists of files the data partition)
- write second program - take a list
- delete one of the program and make third list.

Actually, you could that by yourself in 30 min ;).

#339 OFFLINE   cigar95

cigar95

    Legend

  • Registered
  • 135 posts
Joined: May 23, 2008

Posted 08 July 2011 - 12:55 PM

Sounds like a good idea - although since my DVR is at home and the computer where I'd be doing the UNIX stuff is at work, each step would take a day.

I've never bothered to look at what's on the smallest partition, just the two larger ones - but that might indeed have some important stuff.

I've also never tried to mount the other two partitions separately - just mounted them together. Not sure if a real-time partition can be mounted alone.

#340 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,727 posts
  • LocationBay Area
Joined: Jul 25, 2002

Posted 08 July 2011 - 02:14 PM

No, keep proper mounting, with rtdev part.




spam firewall