Jump to content


Welcome to DBSTalk


Sign In 

Create Account
Welcome to DBSTalk. Our community covers all aspects of video delivery solutions including: Direct Broadcast Satellite (DBS), Cable Television, and Internet Protocol Television (IPTV). We also have forums to discuss popular television programs, home theater equipment, and internet streaming service providers. Members of our community include experts who can help you solve technical problems, industry professionals, company representatives, and novices who are here to learn.

Like most online communities you must register to view or post in our community. Sign-up is a free and simple process that requires minimal information. Be a part of our community by signing in or creating an account. The Digital Bit Stream starts here!
  • Reply to existing topics or start a discussion of your own
  • Subscribe to topics and forums and get email updates
  • Send private personal messages (PM) to other forum members
  • Customize your profile page and make new friends
 
Guest Message by DevFuse

Photo
- - - - -

3TB and 4TB Drives. The Danger!


  • Please log in to reply
192 replies to this topic

#161 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,929 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 07 October 2012 - 09:08 AM

Oh, yeah. I forgot to note - WinXP Pro SP3 working fine with 4K sector size 3 TB drive in GoFlex enclosure.

...Ads Help To Support This SIte...

#162 OFFLINE   JonW

JonW

    Icon

  • Registered
  • 526 posts
Joined: Dec 21, 2006

Posted 07 October 2012 - 09:09 AM

300GB was nice back when we were recording SD and that could hold 200+ hours, but if you record primarily HD, it fills up real fast. 3TB is actually a reasonable number for a HD DVR in my book.

Alas, another problem is that the DVRs weren't designed with that much space in mind. It's not surprising that an MRV'd DVR with a combined playlist is going to get sluggish when those playlists can hold 10x the data originally intended.

#163 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,929 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 07 October 2012 - 09:27 AM

300GB was nice back when we were recording SD and that could hold 200+ hours, but if you record primarily HD, it fills up real fast. 3TB is actually a reasonable number for a HD DVR in my book.

Alas, another problem is that the DVRs weren't designed with that much space in mind. It's not surprising that an MRV'd DVR with a combined playlist is going to get sluggish when those playlists can hold 10x the data originally intended.


I wouldn't say that blindly (but I would be glad to see blueprints of FW) at least what I saw: they does employ DB approach for store and update EPG and system info ...

#164 OFFLINE   Rich

Rich

    Hall Of Fame

  • Topic Starter
  • DBSTalk Club
  • 20,930 posts
  • LocationPiscataway, NJ
Joined: Feb 22, 2007

Posted 07 October 2012 - 11:13 AM

You're assuming that if it becomes "supported" that it will be changed in such a way as to disable anything other than the specific device they are "supporting".

I don't think that's likely. More likely they would support the device(s) they have blessed and the rest would remain as today - works but not supported.


I'm afraid of D* taking over the external function. They haven't "blessed" any HDDs or external devices. They made a mistake by suggesting the Seagate eSATA and were right about suggesting the WD device. Aside from that, they've been quiet. The Seagate bombed, by the way. I just hope they leave the choices of what we use to us and don't try to shove something we don't want down our throats.

Rich

#165 OFFLINE   Rich

Rich

    Hall Of Fame

  • Topic Starter
  • DBSTalk Club
  • 20,930 posts
  • LocationPiscataway, NJ
Joined: Feb 22, 2007

Posted 07 October 2012 - 11:21 AM

300GB was nice back when we were recording SD and that could hold 200+ hours, but if you record primarily HD, it fills up real fast. 3TB is actually a reasonable number for a HD DVR in my book.


For someone with one or two HRs, a 3TB drive would seem to be sufficient. But with many more HRs, 2TBs are more than sufficient. The drawback to the 3TB drives is losing the HR that recorded its content. Then it becomes a horror story.

Alas, another problem is that the DVRs weren't designed with that much space in mind. It's not surprising that an MRV'd DVR with a combined playlist is going to get sluggish when those playlists can hold 10x the data originally intended.


I've got a huge UPL and my 24-500s handle that UPL just fine. Obviously, I don't have 10 times the capacity of my eleven 2TB drives on the UPL, that would be a bit much.

Rich

#166 OFFLINE   bobcamp1

bobcamp1

    Icon

  • Registered
  • 895 posts
Joined: Nov 08, 2007

Posted 07 October 2012 - 12:00 PM

Oh, yeah. I forgot to note - WinXP Pro SP3 working fine with 4K sector size 3 TB drive in GoFlex enclosure.


That's because XP supports 4k logical sectors for a *secondary* USB drive. Take the drive out of the enclosure and connect it directly to SATA and see what happens. My motherboard thinks a 3TB hard drive is only 800 GB (the capacity overflowed: 3 TB - 2.2 TB = 800 GB).

The drive inside the GoFlex enclosure has physical 512 bytes/sector, yet has 4k logical sectors in order to get around the MBR limit. That's a hardware modification -- you've added a special hard disk controller to get around the limit. How would D* put that inside millions of existing DVRs?

There are also other third-party drivers that let XP support GPT. But that hard drive can't contain the system partition, because that driver can't load until the kernel is loaded.

If people have tried it with an HR2x and it doesn't work, then the OS they have (or the chipset they have) doesn't support it. I don't think they're changing code just to add support to something that's not officially supported.

#167 OFFLINE   bobcamp1

bobcamp1

    Icon

  • Registered
  • 895 posts
Joined: Nov 08, 2007

Posted 07 October 2012 - 12:14 PM

300GB was nice back when we were recording SD and that could hold 200+ hours, but if you record primarily HD, it fills up real fast. 3TB is actually a reasonable number for a HD DVR in my book.

Alas, another problem is that the DVRs weren't designed with that much space in mind. It's not surprising that an MRV'd DVR with a combined playlist is going to get sluggish when those playlists can hold 10x the data originally intended.


That happens with Tivos. When I increased the size of my Series 1's hard drive from 20 GB to 120 GB, and I had a lot of shows saved on it, it would take 8 seconds to bring up the playlist. Deleting a bunch of shows sped it right back up. Trick play was not affected. But that 54 MHz processor and 16 MB of RAM just couldn't process that gigantic database quickly. The same thing happens with the Series 4, but to a lesser extent.

#168 OFFLINE   inkahauts

inkahauts

    Hall Of Fame

  • DBSTalk Club
  • 15,441 posts
Joined: Nov 13, 2006

Posted 07 October 2012 - 01:55 PM

300GB was nice back when we were recording SD and that could hold 200+ hours, but if you record primarily HD, it fills up real fast. 3TB is actually a reasonable number for a HD DVR in my book.

Alas, another problem is that the DVRs weren't designed with that much space in mind. It's not surprising that an MRV'd DVR with a combined playlist is going to get sluggish when those playlists can hold 10x the data originally intended.


My HR24 and my HR34 do not show any signs of speed changes based on the drive being empty or completely full, I've checked, they always work at the same speeds, and that's with 2tb drives. My hr21s slow down a little, which Leeds me to believe that they are just underpowered, since they are slow anyway, even compared to an HR20.

#169 OFFLINE   inkahauts

inkahauts

    Hall Of Fame

  • DBSTalk Club
  • 15,441 posts
Joined: Nov 13, 2006

Posted 07 October 2012 - 01:57 PM

For someone with one or two HRs, a 3TB drive would seem to be sufficient. But with many more HRs, 2TBs are more than sufficient. The drawback to the 3TB drives is losing the HR that recorded its content. Then it becomes a horror story.

I've got a huge UPL and my 24-500s handle that UPL just fine. Obviously, I don't have 10 times the capacity of my eleven 2TB drives on the UPL, that would be a bit much.

Rich


I'd bet for 99% of DirecTV subs though, the only reason to have a 3tb drive would be because they have a lot of people recording different things with little overlap, so a failure would only cause everyone to lose a little bit of programing, and not everyone to lose 3tb worth of recordings.

#170 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,929 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 07 October 2012 - 01:58 PM

That's because XP supports 4k logical sectors for a *secondary* USB drive. Take the drive out of the enclosure and connect it directly to SATA and see what happens. My motherboard thinks a 3TB hard drive is only 800 GB (the capacity overflowed: 3 TB - 2.2 TB = 800 GB).

The drive inside the GoFlex enclosure has physical 512 bytes/sector, yet has 4k logical sectors in order to get around the MBR limit. That's a hardware modification -- you've added a special hard disk controller to get around the limit. How would D* put that inside millions of existing DVRs?

There are also other third-party drivers that let XP support GPT. But that hard drive can't contain the system partition, because that driver can't load until the kernel is loaded.

If people have tried it with an HR2x and it doesn't work, then the OS they have (or the chipset they have) doesn't support it. I don't think they're changing code just to add support to something that's not officially supported.

You got it backwards:
'hdparm -I' knows that this is a native 4k disk:
       Model Number:       ST3000DM001-9YN166
       Logical  Sector size:                   512 bytes
       Physical Sector size:                  4096 bytes
       Logical Sector-0 offset:                  0 bytes
       device size with M = 1024*1024:     2861588 MBytes
       device size with M = 1000*1000:     3000592 MBytes (3000 GB)

Here is good reading about 4K sectors, emulation etc : http://www.seagate.c...ives-master-ti/

Edited by P Smith, 07 October 2012 - 04:41 PM.


#171 OFFLINE   inkahauts

inkahauts

    Hall Of Fame

  • DBSTalk Club
  • 15,441 posts
Joined: Nov 13, 2006

Posted 07 October 2012 - 02:01 PM

This thread is funny. We have so many ideas, and DirecTV must just laugh all day long because all of our wants in this regard are probably shared by less than10k subs, making all these ideas of how to provide redundancy not worth the time to even consider for them, especially when a workaround exists, which I think rich is taking advantage of to its fullest. :)

#172 OFFLINE   unixguru

unixguru

    Godfather

  • Registered
  • 579 posts
Joined: Jul 09, 2007

Posted 09 October 2012 - 05:55 PM

This thread is funny. We have so many ideas, and DirecTV must just laugh all day long because all of our wants in this regard are probably shared by less than10k subs, making all these ideas of how to provide redundancy not worth the time to even consider for them, especially when a workaround exists, which I think rich is taking advantage of to its fullest. :)


About the same for OTA yet they have a product for that.

The problem is a catch-22. The "average consumer" doesn't have a clue about what might happen (many don't even backup their computers). And most won't ever see a hardware failure. But when someone gets hit, even more so now with a "Home Media Server" and likely pending storage increase, it's gonna hurt. Until they get hit they won't know to ask for solutions. And with no official product available to solve it they aren't alerted to the possibilities.

You can bet that nobody at DTV or a retailer is going to tell a customer what will happen if a DVR fails unless they are directly asked about it. They have nothing to "up sell" and they sure don't want to advertise their weaknesses.

Consider if just 1 company, cable, DISH, whatever, starts selling a more resilient device, even as an optional premium product. Then they will be explaining to consumers why they might want it. This will create demand for a product that doesn't currently (officially) exist. Because people like reliable products. Then all the sudden all the competitors will have to offer a more reliable product. Why be the follower? Try leading for a change.

The workaround is a PITA for most.

#173 OFFLINE   unixguru

unixguru

    Godfather

  • Registered
  • 579 posts
Joined: Jul 09, 2007

Posted 09 October 2012 - 06:19 PM

I'd bet for 99% of DirecTV subs though, the only reason to have a 3tb drive would be because they have a lot of people recording different things with little overlap, so a failure would only cause everyone to lose a little bit of programing, and not everyone to lose 3tb worth of recordings.


So the fact that everyone in my family has to share the pain is going to make it less?

Sure not the way my family works. I'd be the one getting the mob of "customer" complaints. They can lynch me. :lol:

This is no different than a home computer network. Our server has RAID storage too (and backups).

It's all a no-brainer. These are services that should be as reliable as anything else in the house. Would a similar failure rate be acceptable for a furnace, air conditioner, water heater, stove, refrigerator, freezer, microwave, ...? Those are all more reliable and to top it off one can get a service guy here with parts really quickly and when they are done it's back to full operation. May not even lose food. Lose a freezer full of food because of slow service and it's the same kind of $$ loss.

I know it's just TV. If I lost 2 months worth of a series where the story evolves each episode then I pretty much lost the whole season. Total value is comparable to a freezer full of food. With DTV I know that a failure means that I lose it all regardless of how fast the service.

#174 OFFLINE   bobcamp1

bobcamp1

    Icon

  • Registered
  • 895 posts
Joined: Nov 08, 2007

Posted 10 October 2012 - 07:45 AM

You got it backwards:

'hdparm -I' knows that this is a native 4k disk:
       Model Number:       ST3000DM001-9YN166
       Logical  Sector size:                   512 bytes
       Physical Sector size:                  4096 bytes
       Logical Sector-0 offset:                  0 bytes
       device size with M = 1024*1024:     2861588 MBytes
       device size with M = 1000*1000:     3000592 MBytes (3000 GB)

Here is good reading about 4K sectors, emulation etc : http://www.seagate.c...ives-master-ti/


No, I'm (almost) right. I should have made it more clear that the controller is doing the emulation of the 4k logical sectors. The drive is definitely 512 bytes/sector. Do you have the 3 TB version of this drive?

http://www.everythin...rive-21441.html

Scroll down to "Breaking the 2TB Limit".

In theory, if such a device exists for eSATA instead of USB, then you might be able to use it. 4k logical sector support has been in Linux for quite a while. But the capacity rollover I described above (with my old motherboard) could still be an issue.

#175 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,929 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 10 October 2012 - 09:31 AM

You're still not get it right. You just ignoring data what is posted; BTW, I got the data from MY OWN DRIVE.

Please read original SEAGATE docs, not the some user like you posts (if you will re-read his explanation again you could get grasp of 4k native size (and 512e for OS concept) properly) . And lets finish the off-topic posting.

#176 OFFLINE   Diana C

Diana C

    Hall Of Fame

  • DBSTalk Club
  • 1,876 posts
  • LocationNew Jersey
Joined: Mar 30, 2007

Posted 10 October 2012 - 10:43 AM

You got it backwards:

'hdparm -I' knows that this is a native 4k disk:
       Model Number:       ST3000DM001-9YN166
       Logical  Sector size:                   512 bytes
       Physical Sector size:                  4096 bytes
       Logical Sector-0 offset:                  0 bytes
       device size with M = 1024*1024:     2861588 MBytes
       device size with M = 1000*1000:     3000592 MBytes (3000 GB)

Here is good reading about 4K sectors, emulation etc : http://www.seagate.c...ives-master-ti/


Pardon my butting in on this argument, but what you are showing here is a AF drive in 512e mode. That's fine...this is required for a lots of software (like backup/restore programs) that interact with the drive at a sector level and are not 4K sector aware.

But I am unclear about what that has to do with the original issue of MBR vs. GPT. I guess you could have a MBR drive, using 4k logical and physical sectors exceed the 2.2TB limit, but both your hardware and your OS (and any direct disk accessing software) would have to understand 4K sectors. Boot from a "traditional" disk and add a driver and you can avoid some of the hardware (BIOS) issues, and maybe you could run via 512e on the storage only drive.

But 512e is not a good solution for a DVR since you will often have the situation of misaligned file boundaries. If one video file uses 3 emulated 512 sectors inside one 4K sector, and then the next file wants to use the other 5, you'd have to read the entire 4K sector into memory, merge the new data with the old, and then rewrite the entire sector. This is bound to happen since the DVR software, by definition in this example, thinks it is running on a 512 byte sector drive. Now imagine doing this for 5 write and 3 read processes at once and you pretty soon get someplace I don't think you want to be.

Sure, there are plenty of ways to get 4K sector drives to work...but given the parameters of a DVR I think you either want it done via 512 byte emulation in some dedicated hardware (i.e. a USB drive, which is how Dish does it) or you want to do it through native 4K sector support in the OS (which is exactly what DirecTV has done in the HR34).

Edited by Diana C, 10 October 2012 - 11:05 AM.
corrected misspelling

Dish Network Customer from 9/1998-11/2001
DirecTV Customer 10/2001 - 7/2014

FiOS TV/TiVo Customer since 6/2014
Moderator, DBSDish.com 1999-2000
Co-Founder and Administrator, DBSForums.com 2000-2006

Current setup:
DirecTV: HR34-700 (1TB) / HR24-100 (1TB) / HR24-500 (1TB) / HR21-700 (320GB) / HR21-100 (1TB) / 2 H25s / C41-500 / SWiM16 / Nomad / CCK

FiOS: 2 Tivo Roamio Pros (6 TB total) / 5 Tivo Minis attached via MOCA


#177 OFFLINE   Rich

Rich

    Hall Of Fame

  • Topic Starter
  • DBSTalk Club
  • 20,930 posts
  • LocationPiscataway, NJ
Joined: Feb 22, 2007

Posted 10 October 2012 - 11:04 AM

About the same for OTA yet they have a product for that.

The problem is a catch-22. The "average consumer" doesn't have a clue about what might happen (many don't even backup their computers). And most won't ever see a hardware failure. But when someone gets hit, even more so now with a "Home Media Server" and likely pending storage increase, it's gonna hurt. Until they get hit they won't know to ask for solutions. And with no official product available to solve it they aren't alerted to the possibilities.

You can bet that nobody at DTV or a retailer is going to tell a customer what will happen if a DVR fails unless they are directly asked about it. They have nothing to "up sell" and they sure don't want to advertise their weaknesses.

Consider if just 1 company, cable, DISH, whatever, starts selling a more resilient device, even as an optional premium product. Then they will be explaining to consumers why they might want it. This will create demand for a product that doesn't currently (officially) exist. Because people like reliable products. Then all the sudden all the competitors will have to offer a more reliable product. Why be the follower? Try leading for a change.

The workaround is a PITA for most.


That workaround was the best I could think of. Yeah, it would be a PITA for most folks. And it was/is expensive. But it works quite well.

Rich

#178 OFFLINE   Diana C

Diana C

    Hall Of Fame

  • DBSTalk Club
  • 1,876 posts
  • LocationNew Jersey
Joined: Mar 30, 2007

Posted 10 October 2012 - 11:10 AM

I'm afraid of D* taking over the external function. They haven't "blessed" any HDDs or external devices. They made a mistake by suggesting the Seagate eSATA and were right about suggesting the WD device. Aside from that, they've been quiet. The Seagate bombed, by the way. I just hope they leave the choices of what we use to us and don't try to shove something we don't want down our throats.

Rich


I think you're 100% correct...there would be "official" external drive enclosures. DirecTV has shown that they like nice, predictable, installation configurations. This is not uncommon...most companies want to make things as cookie-cutter as possible - it is the easiest way to scale an operation. There is no way they would ever support "any old" ESATA drive/enclosure.

Dish Network Customer from 9/1998-11/2001
DirecTV Customer 10/2001 - 7/2014

FiOS TV/TiVo Customer since 6/2014
Moderator, DBSDish.com 1999-2000
Co-Founder and Administrator, DBSForums.com 2000-2006

Current setup:
DirecTV: HR34-700 (1TB) / HR24-100 (1TB) / HR24-500 (1TB) / HR21-700 (320GB) / HR21-100 (1TB) / 2 H25s / C41-500 / SWiM16 / Nomad / CCK

FiOS: 2 Tivo Roamio Pros (6 TB total) / 5 Tivo Minis attached via MOCA


#179 OFFLINE   Rich

Rich

    Hall Of Fame

  • Topic Starter
  • DBSTalk Club
  • 20,930 posts
  • LocationPiscataway, NJ
Joined: Feb 22, 2007

Posted 10 October 2012 - 11:18 AM

So the fact that everyone in my family has to share the pain is going to make it less?

Sure not the way my family works. I'd be the one getting the mob of "customer" complaints. They can lynch me. :lol:


I get all the complaints from my family, too. I hate to see my wife, with fire in her eyes, heading for me with a problem. My son just depends on me to fix everything. My granddaughter comes to me and looks at me with those beautiful puppy dog eyes and asks me to fix what ever isn't working for her.

It's all a no-brainer. These are services that should be as reliable as anything else in the house. Would a similar failure rate be acceptable for a furnace, air conditioner, water heater, stove, refrigerator, freezer, microwave, ...? Those are all more reliable and to top it off one can get a service guy here with parts really quickly and when they are done it's back to full operation. May not even lose food. Lose a freezer full of food because of slow service and it's the same kind of $$ loss.


There you'll get the good old "it's only TV" answer.

I know it's just TV. If I lost 2 months worth of a series where the story evolves each episode then I pretty much lost the whole season. Total value is comparable to a freezer full of food. With DTV I know that a failure means that I lose it all regardless of how fast the service.


At least if you lose a freezer full of food for most reasons, your homeowner's insurance will cover it. Who covers what a person with 4TBs of content loses if his 34 fails?

I agree with everything you've said. But, we're gonna be told we're in the minority and we don't matter. Been thru this many times and the answer's always the same. We are anomalies. Nobody but a few of us take advantage of the eSATA function. Not enough people care. If all those answers are valid, why did D* allow larger external HDDs for the 34s?

Rich

#180 OFFLINE   Rich

Rich

    Hall Of Fame

  • Topic Starter
  • DBSTalk Club
  • 20,930 posts
  • LocationPiscataway, NJ
Joined: Feb 22, 2007

Posted 10 October 2012 - 11:23 AM

I think you're 100% correct...there would be "official" external drive enclosures. DirecTV has shown that they like nice, predictable, installation configurations. This is not uncommon...most companies want to make things as cookie-cutter as possible - it is the easiest way to scale an operation. There is no way they would ever support "any old" ESATA drive/enclosure.


We've gone this long without their overt support, I'd like to see it stay that way. They have improved the eSATA function over the years and for that I salute them.

Rich




Protected By... spam firewall...And...