Jump to content


Welcome to DBSTalk


Sign In 

Create Account
Welcome to DBSTalk. Our community covers all aspects of video delivery solutions including: Direct Broadcast Satellite (DBS), Cable Television, and Internet Protocol Television (IPTV). We also have forums to discuss popular television programs, home theater equipment, and internet streaming service providers. Members of our community include experts who can help you solve technical problems, industry professionals, company representatives, and novices who are here to learn.

Like most online communities you must register to view or post in our community. Sign-up is a free and simple process that requires minimal information. Be a part of our community by signing in or creating an account. The Digital Bit Stream starts here!
  • Reply to existing topics or start a discussion of your own
  • Subscribe to topics and forums and get email updates
  • Send private personal messages (PM) to other forum members
  • Customize your profile page and make new friends
 
Guest Message by DevFuse

Photo
- - - - -

3TB and 4TB Drives. The Danger!


  • Please log in to reply
192 replies to this topic

#141 OFFLINE   RunnerFL

RunnerFL

    Hall Of Fame

  • DBSTalk Club
  • 15,832 posts
Joined: Jan 04, 2006

Posted 05 October 2012 - 04:06 PM

Don't understand that. I know what a JBOD is but I don't understand what you're doing with all of the setups. In short, I'm lost. Whoosh.

Rich


I tested each setup out using 2 RAID enclosures that I have to verify that the HR34 was in fact using the entire drive. One enclosure holds 2 drives. With that enclosure I tested 3TB RAID 1 (2x3TB drives), 6TB JBOD (2x3TB drives) and 4TB JBOD (2x2TB drives). The other enclosure I have holds 4 drives and supports RAID 5. With it I tested 12TB JBOD (4x3TB Drives) and I settled on using 9TB RAID 5 (4x3TB Drives).
THIS SPACE FOR RENT

...Ads Help To Support This SIte...

#142 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,868 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 05 October 2012 - 04:13 PM

I'm relatively certain that this is a false statement.


Then you'll eat a crow ! Ready ?

#143 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,868 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 05 October 2012 - 04:15 PM

The MBR can only support 2^32 unique LBAs, because it only supports 32-bit values for them.

2^32 * 512 bytes/sector = 2.2 TB. No partition can go past the 2.2 TB point on the hard drive when using MBR.

You need to replace the MBR with a GPT, update all the built-in disk tools (ex. gdisk not fdisk) accordingly, and whatever little misc. things the software and/or firmware might need to get it to work. It's not trivial, especially when trying to do it remotely for millions of customers with existing boxes full of recordings they'd like to keep.

It's a lot easier to do what D* did -- create a new branch of software for a new device and have that device support it from its initial release.

It was thought that advanced format drives would set this limitation to 17.6 TB, but to make them backward compatible they emulated 512 bytes/sector. Each one of those emulated sectors needs a unique LBA entry, and you have the same exact problem.

The THR22-100 (and all Tivos) have the same problem, as the Apple Partition Manager is also limited to 32-bit LBA entries.


OK. I would use TiB when giving the number, but not everyone could separate computer's and marketing count of bytes. :eek2:

Now turn on your math skills again and do same for newest HDD which has 4K sector's size.
Continue counting ... what is your result ?


EDIT: Done ? So you would still using MBR and it would not required that dramatic changes in FW.

Edited by P Smith, 05 October 2012 - 04:24 PM.


#144 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,868 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 05 October 2012 - 04:48 PM

I tested each setup out using 2 RAID enclosures that I have to verify that the HR34 was in fact using the entire drive. One enclosure holds 2 drives. With that enclosure I tested 3TB RAID 1 (2x3TB drives), 6TB JBOD (2x3TB drives) and 4TB JBOD (2x2TB drives). The other enclosure I have holds 4 drives and supports RAID 5. With it I tested 12TB JBOD (4x3TB Drives) and I settled on using 9TB RAID 5 (4x3TB Drives).


Can't find your post where the big drives/volumes partitions posted ...

#145 OFFLINE   RunnerFL

RunnerFL

    Hall Of Fame

  • DBSTalk Club
  • 15,832 posts
Joined: Jan 04, 2006

Posted 05 October 2012 - 06:31 PM

Can't find your post where the big drives/volumes partitions posted ...


Well here's where I at least posted the size of DirecTV's partition for each setup.

http://www.dbstalk.c...3&postcount=174

You can search for the rest.
THIS SPACE FOR RENT

#146 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,868 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 05 October 2012 - 06:58 PM

I did expect fdisk output ...

#147 OFFLINE   RunnerFL

RunnerFL

    Hall Of Fame

  • DBSTalk Club
  • 15,832 posts
Joined: Jan 04, 2006

Posted 05 October 2012 - 07:09 PM

I did expect fdisk output ...


Then you should do some testing.

Oh, and fdisk won't work with GPT. But you already knew that, right?
THIS SPACE FOR RENT

#148 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,868 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 06 October 2012 - 12:20 AM

GPT ? I don't remember if it was mentioned. For sure 4K drives could work with MBR just fine. As it done by dish for support big EHD drives.

#149 OFFLINE   unixguru

unixguru

    Godfather

  • Registered
  • 579 posts
Joined: Jul 09, 2007

Posted 06 October 2012 - 07:29 AM

Well written post. What I'd REALLY like is the ability to use a NAS on my network to backup/store stuff on the HR34.


Conceptually that would be really nice.

From a practical perspective it creates a lot of issues. NAS has to be carefully engineered in a large enterprise environment. Even there it's not suitable for all purposes.

I've been doing high-end enterprise-scale software engineering for 30 years (storage focus the last 12 years). If I were developing DTV's software I would never consider supporting NAS. Would be asking for disaster.

#150 OFFLINE   unixguru

unixguru

    Godfather

  • Registered
  • 579 posts
Joined: Jul 09, 2007

Posted 06 October 2012 - 07:51 AM

The fact is there isn't a demand because your average consumer just doesn't need it. If HD storage space was a primary driver of consumers needs for a DVR then cable companies would have increased their HD years ago.

This board is full of power users and even here only a small % use them. I'd be interested to see what % of DISH subs use this feature and it has everything that people here are asking for. I still bet it's a small % of DVR users that have the ability to do so.


That's the answer I've been getting for years. I believe it. I think it's typical of the Great American Herd to ignore the good things technology can provide. Or just not understand it. I don't think the eSATA function is a "good" thing, I think (hell, I know) it's a "great" thing.

But, I do believe you're correct in your assumptions.


Most companies belong to another kind of herd - the Good Enough Herd. In the technology realm the most successful member of that herd was/is Micro$oft. Most companies like that model and follow it like a religion. DTV is a full member of that herd.

If you don't have to make something better then the greater herd will just keep sending you money.

Another company has emerged that has a different approach. Innovate and provide something that consumers will want once they see it. Apple.

Proven that it's better for consumers and the company. Way better. Think I recently read that the iPhone is bigger revenue than all of Micro$oft!

Whether you prefer the products of one or the other of the mentioned companies, it doesn't matter. The comparative financial performance is a direct representation of how the majority of consumers have voted.

The chances of DTV adopting that model are... well, very slim.

#151 OFFLINE   unixguru

unixguru

    Godfather

  • Registered
  • 579 posts
Joined: Jul 09, 2007

Posted 06 October 2012 - 08:03 AM

Other way around. BSD is a variant of AT&T Unix and Linux came later as a way to get a PD form that was very similar.


Apologize for the off topic... Unix History

Linux is not a direct derivative of any Unix. It's a Unix-like clone. Except of course for the stealing of code that happened in later years - and was never enforced because of the details of who currently owned what rights to Unix.

However, much of that has been replaced by GNU programs which I guess is the Unix variant of Linux.


This is incorrect. Those with interest can see Wikipedia for the facts.

#152 OFFLINE   Rich

Rich

    Hall Of Fame

  • Topic Starter
  • DBSTalk Club
  • 20,635 posts
  • LocationPiscataway, NJ
Joined: Feb 22, 2007

Posted 06 October 2012 - 08:23 AM

Since we are talking about low consumer demand, why did the 2GB limit get removed for the HR34? If the logic is that DTV never does something unless there is a lot of demand then that change would never have been made. I've never met a Product Manager yet that was in favor of spending any time/money on upgrading core technology just to stay current with technology trends.


I think it finally dawned on some intelligent person at D* that a 1TB drive hooked up to five tuners would be a problem.

It sure wasn't done to make the few external storage geeks happy since it's not supported at all.


While the eSATA function is not "supported", they do make changes at times to it. I'm still against it being supported. For many reasons. Mainly, because I like the idea that I can use any HDD I want to. I really don't want to be limited in my choices.

That means that larger internal storage is coming - because there is demand for it. It's a duh moment for a device marketed as a "Home Media Server".


Yup. Just putting all your eggs in a larger basket.

The gist of this thread, whether internal or external storage, is that as the capacity rises people will be utilizing it and the reaction to a device failure is going to be hotter.


Yup, just wait till the first person has a catastrophic failure of an HR such as the 34 with a four TB HDD on it.

I'm sure the average consumer has not a care in the world about RAID storage either. They don't even have a clue that such a thing is available. But I would be surprised if any consumer was happy about a DTV DVR failure! If they were aware that technology existed that prevented loss from a drive failure they would WANT IT. Likewise if they knew their programming could be easily transferred if the box died, they would WANT IT. Neither of these is difficult.


Exactly.

What is a moderate irritation today is going to be a whole world of hurt when an "average consumer" finds out that >2TB of programming for their entire household is GONE.


Should be interesting watching someone go ballistic. Always is... :lol:

It's just lame that a "Home Media Server" was even brought to market without both of these issues being addressed. It should have had at least 2 (RAID1) "storage cartridges" (ie consumer removable disk magazines). If Joe Consumer (and Joe Installer) can handle inserting and removing CDs and DVDs they can sure handle a disk cartridge. I can recall my 5-year-old son swapping Nintendo cartridges!


Pretty normal for D* to release equipment that's not well thought out. Then fix it after torturing a bunch of people.

The obvious solution now is for DTV to come out with an HR34 model without internal storage and sell something similar to a TenBox (only with removable drives). And fix the program locking.


That makes sense.

The extra cost always comes up when talking about RAID. It's a short-sighted perspective. How much programming is recorded on your DVR that you haven't watched yet? How much is it worth to you - or more like how much did you already pay for it? And how much of it won't be rebroadcast anytime soon? It doesn't take long before the insurance offered by RAID is dirt cheap. For me, I've had 2 drives fail in the last 5 years. The programming that was saved, not to mention the aggravation avoided, was easily worth the expense of RAID.


There are other options other than RAID, but whatever works for someone is good. Having 12 active HRs works really well. I haven't missed a show in a long time due to failure of an HR or an HDD.

Good post!

Rich

#153 OFFLINE   unixguru

unixguru

    Godfather

  • Registered
  • 579 posts
Joined: Jul 09, 2007

Posted 06 October 2012 - 08:35 AM

For several years HP sold a PC with a drive bay where users could plug in a USB drive and swap it out at will. It looked great and worked well, but HP stopped doing it because the plug-in drives were of course proprietary, undersized and expensive.

I love your idea of extending this to the HMC, and either allowing users to swap drives or if nothing else, giving techs spare drives to plug in during a service call. It's plug-and-play at its simplest.

This would require a much larger chassis, though, and likely will raise the cost of the devices, so unfortunately I don't see it happening. Would be really nice though ...


The expense issue is pretty much gone. SATA drives are, by definition, hot-swappable. The bare drive connectors can stick right out the back of the cartridge. No electronics glue of any kind. The only reason a cartridge is needed at all is to protect the bare circuit board on the one side of the drive. A simple piece of plastic with holes for ventilation is all that is required to make a consumer-friendly cartridge.

Cheap enclosures like the Sans Digital already have a cartridge that is almost what is needed. They are just a U-shaped piece of plastic with maybe a small metal rail on each side. Most of them don't have any kind of cover over the drive circuit board. These are sold to consumers so the assumption apparently is that consumers are smart enough to avoid static discharge to the circuit board :) In any event, adding a piece of plastic with holes in it would add only pennies to the cost.

The RDX cartridge is a current example. They use 2.5" drives. The mobile drive has some extra shock-tolerance features (retracted heads). No reason DTV couldn't do something similar with 3.5" drives and rate them with less shock resistance - after all, the usage scenario is completely different.

It would increase the price to do it internally. Bigger case, power supply, fan. But not a huge amount. Again, look at the price of a Sans Digital 2 drive enclosure. The HR34 is a premium device.

But we don't need a new generation of HR34. Just make something like a TenBox with drive cartridges. The AM21 was a great solution for OTA for the subset of people who wanted it. External RAID would be the same for storage space and reliability. The development and manufacturing cost would be minimal; nothing special, no software to develop, etc.

Lots of enterprise arrays have an LED on each drive slot that lights up when the drive is defective. Shouldn't even need a truck roll for this - just ship out a new cartridge and a return label. Lexmark laser printer cartridges, for example, have used that approach forever.

#154 OFFLINE   bobcamp1

bobcamp1

    Icon

  • Registered
  • 895 posts
Joined: Nov 08, 2007

Posted 06 October 2012 - 08:53 AM

OK. I would use TiB when giving the number, but not everyone could separate computer's and marketing count of bytes. :eek2:

Now turn on your math skills again and do same for newest HDD which has 4K sector's size.
Continue counting ... what is your result ?


EDIT: Done ? So you would still using MBR and it would not required that dramatic changes in FW.


OK, I'm counting the first six physical consecutive sectors on a typical Advanced Format drive....

0, 8, 16, 24, 32, 40, ....

Hmmm... I've only counted six sectors yet the number I need to use to access that sector (its LBA) is already at 40. I have to count by 8's. The number '8' comes from 4096/512. The hard drives physical sector size is 4k, but its logical sector size is still stuck at 512 bytes/sector. And the logical sector size is what the OS uses to access it. So the MBR limitation still exists.

That's why these drives are also called "512e". They emulate everything, good and bad, from the old style 512 bytes/sector drives.

I don't expect the emulation to go on forever. There's already "4k native" (4Kn) external USB drives. But it's going to be a while before they become popular because OSs are just starting to support them. For example, Windows 7 doesn't support it but Windows 7 SP1 does (which is why you need to slipstream SP1 into your bootable Windows 7 DVD).

I have no idea if the version of Linux in the HR34 supports 512e only or if it can do 4Kn. Does anyone know the model of the hard drive in it? Or you could plug in a 4Kn eSATA drive in and see if the HR34 can use it.

#155 OFFLINE   unixguru

unixguru

    Godfather

  • Registered
  • 579 posts
Joined: Jul 09, 2007

Posted 06 October 2012 - 09:04 AM

There are other options other than RAID, but whatever works for someone is good. Having 12 active HRs works really well. I haven't missed a show in a long time due to failure of an HR or an HDD.


Yes, redundancy at the entire device level works - and is better actually given the inability to move storage/programs to another device.

But it adds a lot of operational overhead. MRV is half-baked. Cannot do all operations from any device remotely. No ability to say device 2 mirrors device 1 (i.e. records everything that the other does). Etc.

I suppose they could, in the future, allow more than one HR34 in a house. Allow one device to mirror the other (actually needs to be a 2-way mirror so that when one fails and is replaced the new one is automatically brought back into sync). Let the C31 automatically fallback to the alternate upon failure...

(That is actually similar to a standard failover cluster in the enterprise world. There one uses one set of RAID storage connected via a redundant storage network [fibre channel, iscsi, fcoe, etc] with 2 or more hosts running cluster software. Complicated and EXPENSIVE!)

That's a much more expensive approach: purchasing the 2nd device, paying the additional receiver fee every month, developing and supporting a lot more software, etc. Movable RAID is a much cheaper solution.

#156 OFFLINE   unixguru

unixguru

    Godfather

  • Registered
  • 579 posts
Joined: Jul 09, 2007

Posted 06 October 2012 - 09:21 AM

While the eSATA function is not "supported", they do make changes at times to it. I'm still against it being supported. For many reasons. Mainly, because I like the idea that I can use any HDD I want to. I really don't want to be limited in my choices.


You're assuming that if it becomes "supported" that it will be changed in such a way as to disable anything other than the specific device they are "supporting".

I don't think that's likely. More likely they would support the device(s) they have blessed and the rest would remain as today - works but not supported.

#157 OFFLINE   P Smith

P Smith

    Mr. FixAnything

  • Registered
  • 19,868 posts
  • LocationMediterranean Sea
Joined: Jul 25, 2002

Posted 06 October 2012 - 09:31 AM

OK, I'm counting the first six physical consecutive sectors on a typical Advanced Format drive....

0, 8, 16, 24, 32, 40, ....

Hmmm... I've only counted six sectors yet the number I need to use to access that sector (its LBA) is already at 40. I have to count by 8's. The number '8' comes from 4096/512. The hard drives physical sector size is 4k, but its logical sector size is still stuck at 512 bytes/sector. And the logical sector size is what the OS uses to access it. So the MBR limitation still exists.

That's why these drives are also called "512e". They emulate everything, good and bad, from the old style 512 bytes/sector drives.

I don't expect the emulation to go on forever. There's already "4k native" (4Kn) external USB drives. But it's going to be a while before they become popular because OSs are just starting to support them. For example, Windows 7 doesn't support it but Windows 7 SP1 does (which is why you need to slipstream SP1 into your bootable Windows 7 DVD).

I have no idea if the version of Linux in the HR34 supports 512e only or if it can do 4Kn. Does anyone know the model of the hard drive in it? Or you could plug in a 4Kn eSATA drive in and see if the HR34 can use it.

I'm talking about support of 4k drives not emulation.
Using MBR and increasing space by using same vector: 32 bits.
If you continue live in a past, then here is simple answer: 2.2 TB x 8 = 17.6 TB.
That is new limit for using old MBR. See my other post about _real_ partitioning of 3 TB EHD by dish.

#158 OFFLINE   harsh

harsh

    Beware the Attack Basset

  • Registered
  • 18,739 posts
  • LocationSalem, OR
Joined: Jun 14, 2003

Posted 06 October 2012 - 12:43 PM

Then you'll eat a crow ! Ready ?

I'm ready for some detailed factual information as opposed to your verbally confusing teases and hints.

Too often we enjoy the comfort of opinion without the discomfort of thought. -- JFK


#159 OFFLINE   RunnerFL

RunnerFL

    Hall Of Fame

  • DBSTalk Club
  • 15,832 posts
Joined: Jan 04, 2006

Posted 06 October 2012 - 02:10 PM

GPT ? I don't remember if it was mentioned. For sure 4K drives could work with MBR just fine. As it done by dish for support big EHD drives.


Well you can trust me or you can try it yourself. You decide.
THIS SPACE FOR RENT

#160 OFFLINE   TBoneit

TBoneit

    Hall Of Fame

  • Registered
  • 2,276 posts
Joined: Jul 27, 2006

Posted 07 October 2012 - 07:58 AM

OK, I'm counting the first six physical consecutive sectors on a typical Advanced Format drive....

0, 8, 16, 24, 32, 40, ....

Hmmm... I've only counted six sectors yet the number I need to use to access that sector (its LBA) is already at 40. I have to count by 8's. The number '8' comes from 4096/512. The hard drives physical sector size is 4k, but its logical sector size is still stuck at 512 bytes/sector. And the logical sector size is what the OS uses to access it. So the MBR limitation still exists.

That's why these drives are also called "512e". They emulate everything, good and bad, from the old style 512 bytes/sector drives.

I don't expect the emulation to go on forever. There's already "4k native" (4Kn) external USB drives. But it's going to be a while before they become popular because OSs are just starting to support them. For example, Windows 7 doesn't support it but Windows 7 SP1 does (which is why you need to slipstream SP1 into your bootable Windows 7 DVD).

I have no idea if the version of Linux in the HR34 supports 512e only or if it can do 4Kn. Does anyone know the model of the hard drive in it? Or you could plug in a 4Kn eSATA drive in and see if the HR34 can use it.


I don't see why you would need to slipstream SP1, Who would use a 4Tb boot drive?
My computers boot Drives are 120Gb, 160Gb and 300Gb. All SSD The two laptops with the smaller drives, They are more than large enough. The Desktop has 2Tb data drives Since I use it to capture, convert and Author BluRay. And I use 2Tb USB3 drives to back up from the desktop for their speed.

TB
Remember when your kids were the TV set's remote control?




Protected By... spam firewall...And...