Question for OS experts- Max file size for win XP

[DonLandis]

Normally I don't get into a project like this but I'm working on a long form video and tried to create an AVI file for a multi hour program. It's estimated to be about 4 hours long. The first test run to render completed an AVI file that exceeded 100Gb using MS Video for windows codec. This wasn't even the entire project but just a test run. The result is that the file was not playable in any of my avi players, not to mention would not load into the editor for further editing.

I did another test run and as long as I kept the file size below 50Gb it would play fine as well as load into the timeline.

My plan of action is to split up the project into smaller AVI chunks that play and then load them all into the editor, do my final editing and then render to a compressed file format for loading on a DVD at a bit rate that will fit.

So I now have 6 45Gb AVI files to begin editing. They all play fine.



Is there some limitation on AVI file size I don't know about? Is this an operating system limitation that screws up the codec? The file does save, it's just not playable. So this tells me it's not a win XP limitation.

[Bogy]

I think you should ask Don Landis. He's the expert on stuff like this. :)

[P Smith]

2 TB for NTFS v4 or v5: http://ntfs.com/ntfs_vs_fat.htm

Check the AVI file size limit with software manufacturer.

[DonLandis]

I thought so. So you might think it is a file size on AVI just within the software?

I've had to do some more traditional work today so I had to put the big job aside for now. The application I was using for this is 3D Album and it takes still images and animates them within a 3D canned image sequence. It may be an issue of number of pictures loaded more than file size. I discovered one of the projects actually completed with 455 images and played fine while another at 600 Pictures but holding less on the screen yielding a smaller file size would not play. I'm going to follow your suggestion and check out the application's limits not just for AVI size but also the number of pictures it can do at a time.

[cdru]

The original AVI spec says the complete file, after compression, is to be under 2GB in size. Was the 100GB (or 50GB) the uncompressed version or the compressed final version?

There are workarounds available, such as using segmented AVIs, but your software may or may not support that. You can read a little about it here (http://neuron2.net/LVG/filesize.html).

[DonLandis]

cdru-- I do believe that was an old myth and was based on some mis interpretation of the OS File store at that time. In particular- FAT 16 that had the 2GB limitation. I recall those limitations that proved not to be true once the OS limitation was lifted. I've been doing uncompressed AVI, as well as compressed avi with a variety of codecs in the 6 to 20GB range for years. However, this is the first time I've ever encountered a flaw in the ~50Gb and more range that now seems to be software limitation on the picture count (jpg file count) rather than file size of the avi. There may be a special codec for AVI that has that limitation but none of the ones I use here do. ( DV AVI and MSVideo AVI)


YUP! I just looked at your reference link and that confirms what my recollection of the myth was. Your ref had more detail but I can offer that this was not jkust an AVI thing but all filesizes for the File store system used such as FAT 16, 32 or NTFS.

FYI- All I use is NTFS.

[ntexasdude]

As I recall the 2 gig limitation was due to how the OS wrote bit and bytes into certain sized sectors on the platters. In essence it was due to poor coding by lazy programmers. Am I correct?

[DonLandis]

When formatting the older file systems you did have to be careful depending on what sort of files you were typically saving with your data. The rule changed depending on whether you were saving a lot of small files use lower bytes per sector or single large files, use large bytes per sector and get more capacity. That was a management issue back in the days when we were doing renderings to many smaller image sequence files. TYhen we would record them with something like a Diaquest single frame VCR controller. Today, the computers are so fast you stream full video at full frame rate of 29.97 fps and don't worry about it. But those files are large, so you just use the larger bytres per sector. I don't think there was anything lazy about it as that was just the technology of the day.

[P Smith]

As I recall the 2 gig limitation was due to how the OS wrote bit and bytes into certain sized sectors on the platters. In essence it was due to poor coding by lazy programmers. Am I correct?
Wrong. It was matter of numbers in OS ( FAT bits, LBA, CHS in MBR, etc ).

[CoriBright]

I've certainly had several files of 66gb in size on my largest hard drive... each was about an hour's worth of totally uncompressed DV-AVI. XP Pro with SP2 and all critical updates since. NTFS (of course)

[DonLandis]

Thanks Cori- That convinces me now it must be an issue with the application's ability to handle large number of jpg files on the input side of the editing. Not it's ability to build a large DV AVI file on an NTFS data drive.

I did go through the specs on the software and it doesn't state an image file count limit so it's time to contact the software support people for an answer.


Appreciate everyone's input. Glad it's not an OS limitation.