Question about History, First Runs, and Repeats

[Maverickster]

As we all know, the guide data used in the HR20 is somewhat unreliable with respect to what is and is not a "First Run" and a "Repeat". I know this is not DirecTV's fault (except to the extent they chose the guide data provider), and I know that the exact impact the writer's strike is having on this is still somewhat up for debate.

As a consequence of this, many of us, myself included, select "Both" when setting up our Series Links in order to eliminate the possiblity of missing a "First Run" that is incorrectly labeled/flagged.

In my prior TiVo-owning life, I seem to recall that it had something like a 28 day memory, so that when you selected "First Runs and Repeats" (or whatever "Both" was called), it would get both First Runs and Repeats, but it wouldn't record the exact same show a second time if it had recorded it in the prior 28 days. I always thought this was a nifty little feature.

My HR20 doesn't seem to operate the same way, though. If I have a Series Link set up for Program X and have it set to record "Both", it seems to record Episode 1 each time Episode 1 is run regardless of whether it just recorded Episode 1 the day before or not.

It would seem possible (I'm not a software engineer, so I'm not going to say "easy") for the HR20 to operate the same way as the TiVo did using the History as the reference point. So, for instance, if I set up a Series Link to record "Both" of Program X, it should record all Episodes of Program X UNLESS that Episode is in the History (indicating its been recorded relatively recently -- for those of us with a lot of SL's anyway), in which case it should not record it.

Is this how it's "supposed" to work and my HR20 is just not working that way, or is my HR20 working how it's "supposed" to, and I'm just looking for, essentially, an item I should put on my personal wishlist?

Thanks all!

--Mav

[Earl Bonovich]

Yes the HR20 has a 28 day memory as well... in addition, it will not re-record anything that is on the hard drive.

The 28 day rule will ONLY work, if the specific show is coded correctly.
The HR2* relies on a hidden field in the data: SERIES ID and EPISODE ID

A lot of times that is generic for a show... and thus doesn't help the HR2* in identifing repeates/duplicates.

[MLBurks]

All I know (and sometimes, that's not much) is when I was with Dish, I never had an issue either with the 625 or 722 with first runs or repeats. My programs got recorded exactly how I wanted them to and I never missed a recording. So if Dish can do it, so can D*. That being said, I think that this issue has improved somewhat over the last year or two.

[Maverickster]

Yes the HR20 has a 28 day memory as well... in addition, it will not re-record anything that is on the hard drive.

The 28 day rule will ONLY work, if the specific show is coded correctly.
The HR2* relies on a hidden field in the data: SERIES ID and EPISODE ID

A lot of times that is generic for a show... and thus doesn't help the HR2* in identifing repeates/duplicates.

I see. That's very helpful. It's not a "generic description" issue since it does it with programs that have "full descriptions" (to the extent they're visible, anyway). Take "Top Gear" on BBCA as an example. With SL set to "Both", it has recorded duplicate episodes within a week of each other both of which have the same "full" description (e.g. "(2006) The Boys go to London to do X, Y, and Z"). In fact, if memory serves, it's done this even though one of the episodes is already on the hard drive, resulting in duplicate identical episodes in the List Folder.

If I'm understanding you correctly, I take it this is an extension of "bad" guide data being furnished to DirecTV. Not only is the "guide data" unreliable in terms of first runs/repeats, but it also sometimes has incorrect hidden SERIES ID and EPISODE ID info as well (making the HR20 think that it's recording different programs because the guide data's SERIES ID and EPISODE ID are telling it that they are different when, in fact, they are the same).

Assuming that to be the case, where do I sign the "DirecTV needs to find a new guide data provider" petition?

Thanks a lot Earl.

--Mav

[houskamp]

I've seen the same problem with "eco-tech".. had it set to record all episodes.. I didn't relize they only made 3 at the time and was weekly deleting the same 3 episodes for a month till I finaly deleted the series link..

[gcisko]

Yes the HR20 has a 28 day memory as well... in addition, it will not re-record anything that is on the hard drive.

The 28 day rule will ONLY work, if the specific show is coded correctly.
The HR2* relies on a hidden field in the data: SERIES ID and EPISODE ID

A lot of times that is generic for a show... and thus doesn't help the HR2* in identifing repeates/duplicates.

Are you sure about all this? What I have experienced does not match what you explain. Especially about programs already recorded and on my HR20 hard drive.

Sorry!

I have had series recordings of both American Chopper and the new Dennis Miller sports show (and many on HBO) that would re-record many identical episodes within one week (ditto for shows on WTTW). So the 28 day rule is for the hidden field data. How do you explain that it recorded episodes that were already on my HR20 disk?

And I remember this never happened on my old UltimateTV. So do you have any explaination why the HR2x apparently has this feature but sometimes doesn't work while the UltimatTV never had that problem?

[rudeney]

I have some auto records setup and they are constantly re-recording the same shows. For example, I’m a big Disney buff, so I have mine set to record anything with “Disney World” in the title. I get multiple recordings of the same shows like “Walt Disney World Ultimate 10” or “Walt Disney World Fan’s Guide” at least once a week, sometimes even twice in the same day. This is an HR2x issue and not a D* or guide data issue as I had the same thing setup on my D* UTV and it never recorded duplicates of the show. It knew from the history that it had already recorded the show and would not duplicate it – even when it was broadcast on different channels, like TLC vs. Travel. It’s not a big deal, but it is inconvenient to have to go clean out these recordings when some very simple logic (that obviously already existed in the UTV) would take care of it.

[trainman]

Assuming that to be the case, where do I sign the "DirecTV needs to find a new guide data provider" petition?

Problem is, there are exactly two electronic guide data providers in the U.S.: Tribune Media Services and Gemstar-TV Guide. Back when I had a standalone TiVo that used Tribune data and a cable box that used TV Guide data, I often had the opportunity to compare the two (because the TV Guide data from the cable box would always show up at the beginning of my recordings as the TiVo changed channels). The TV Guide data was noticeably worse overall than the Tribune data, and that's just in terms of having the correct program listed.

Granted, this was the 2001-2004 time frame, so it's possible TV Guide's data has improved.

[Maverickster]

Problem is, there are exactly two electronic guide data providers in the U.S.: Tribune Media Services and Gemstar-TV Guide. Back when I had a standalone TiVo that used Tribune data and a cable box that used TV Guide data, I often had the opportunity to compare the two (because the TV Guide data from the cable box would always show up at the beginning of my recordings as the TiVo changed channels). The TV Guide data was noticeably worse overall than the Tribune data, and that's just in terms of having the correct program listed.

Granted, this was the 2001-2004 time frame, so it's possible TV Guide's data has improved.

Well, if TiVo uses the Tribune data (which I'm not sure is right; I thought TiVo ran their own guide), then this "don't look at us; it's the guide data provider's fault" reason is just a cop out since TiVo managed to do it "right" using the same data, while the HR20 fails fairly miserably in this department (both on the repeat/first run distinction and in the problem discussed in this thread).

Earl, is this a guide problem or an HR20 problem? I suspect I know what your answer is going to be, but I thought I'd ask anyway.

--Mav

[Earl Bonovich]

Are you sure about all this? What I have experienced does not match what you explain. Especially about programs already recorded and on my HR20 hard drive.

Sorry!

I have had series recordings of both American Chopper and the new Dennis Miller sports show (and many on HBO) that would re-record many identical episodes within one week (ditto for shows on WTTW). So the 28 day rule is for the hidden field data. How do you explain that it recorded episodes that were already on my HR20 disk?

And I remember this never happened on my old UltimateTV. So do you have any explaination why the HR2x apparently has this feature but sometimes doesn't work while the UltimatTV never had that problem?


I don't know the inner workings of UltimateTV so as for why it is doing, and how it compensates and works with the guide data... I don't know.

You are also not talking Red-Apples to Red-Apples... the guide data may have been different back when you were using your ultimate TV.


As for am I sure... Yes, that is how it has been explained to me..
And that is how at least in my testing, how it works.

I record several shows that are on multiple times a night, and replayed later in the week... and I only get single records
I do also have American Chopper setup from TLC, and I only get the single records, I do not get multiples.

Do you have them setup for First Run's only? or Both?
Most of my settings are First Run only...

I have American Gladiators from ECL set to BOTH, and that does record the second replay later in the evening.
So it is possible that the BOTH acts as not only recording repeats, but also records duplicates.

[Earl Bonovich]

Well, if TiVo uses the Tribune data (which I'm not sure is right; I thought TiVo ran their own guide), then this "don't look at us; it's the guide data provider's fault" reason is just a cop out since TiVo managed to do it "right" using the same data, while the HR20 fails fairly miserably in this department (both on the repeat/first run distinction and in the problem discussed in this thread).

Earl, is this a guide problem or an HR20 problem? I suspect I know what your answer is going to be, but I thought I'd ask anyway.

--Mav

TiVo has 10 years of manipulation to it's guide data processing to compensate for issues with the guide data... and it is not "perfect" in that manipulation..... There are times that even TiVo can't handle some of the guide data that it is given.

With the guide data it is GIGO Garbage In - Garbage out.

The issues with First Run/Repeat is HIGHLY dependent on the show.

So what show are you seeing an issue with?
What we have seen over the last two years, the "cable" networks are some of the worst with this issue..... not so much the network channels.

I have 41 Series Links, most of them are on Network TV... and I haven't had many issues with First Run vs Repeat (especialy over these last three months where there have been a TON of repeats). I get a random time it records a repeat, but upon investigating... it is typically an episode, that had generic or no detailed guide data to it, so the system was just falling back to "record just in case", let the user delete it if they don't want it.

[sbelmont]

Earl - I had this issue crop up this last week. Never seen it happen before, but on Saturday my wife got 4 recordings of Trading Spaces. All the same episode, all first run, all with the same show description.

One thing I miss from the Tivo was the extra show information when you hit the info button while looking at the show description. One of the things listed was the first air date of the episode. It would be nice to see that feature here also.

[finaldiet]

Question Earl? In my prioritizer list I have about 15-20 shows marked to record. In my to-Do list it shows only 4. Just to see what happens, I re-entered one of the shows to record first run and when I checked TO-DO, nothing there. Will I have to re-enter shows to record

[rudeney]

I don't know the inner workings of UltimateTV so as for why it is doing, and how it compensates and works with the guide data... I don't know.

You are also not talking Red-Apples to Red-Apples... the guide data may have been different back when you were using your ultimate TV.

I had a couple of UTV units running up until last month and still never had the problem. I don’t believe they use the same guide data, though. Either that or the HR2x doesn’t get all the guide data that the UTV was receiving because the UTV had more details – actors, directors, producers, etc.

Do you have them setup for First Run's only? or Both?
Most of my settings are First Run only...

I have American Gladiators from ECL set to BOTH, and that does record the second replay later in the evening.
So it is possible that the BOTH acts as not only recording repeats, but also records duplicates.

Good observation. On the UTV, that didn’t matter, so maybe it does on the HR2x. I know the auto records I have setup are for “both”, so I may change that and see what happens.

[Earl Bonovich]

What show?

Also, after you add programs to your prioritizer, it can be several hours until it populates the ToDo list.

What are the other shows that are not showing up?
Are they first runs only?

[Maverickster]

I have American Gladiators from ECL set to BOTH, and that does record the second replay later in the evening.
So it is possible that the BOTH acts as not only recording repeats, but also records duplicates.

That doesn't make a whole lot of sense. We all know that in practice, there are fairly significant problems with this, but in theory, "First Run" records the first showing of an episode and ignores all repeats, right? If "Both" records First Runs, Repeats, AND Duplicates, then where does the "28 day rule" and the "already on the hard drive rule" come into play? In order for those rules to make any sense whatsoever, then "Both" must mean First Runs and Repeats, but not Duplicates, right? Otherwise those rules would be rules in search of a place to operate.

Now, in practice, "First Run" is problematic option because of questionable data regarding which episodes are first runs and which episodes are repeats; and "Both" is problematic because it's getting First Runs, Repeats, and Duplicates when it's supposed to just be getting First Runs and Repeats.

Are you telling us that "Both" is not a problem because it's SUPPOSED to be getting First Runs, Repeats, AND Duplicates? If so, then when are the "28 day rule" and the "already on the hard drive rule" supposed to apply? If not, and if "Both" is only supposed to get First Runs and Repeats and ignore Duplicates, then why are the "28 day rule" and the "already on the hard drive rule" not working in a lot of cases?

Thanks Earl.

--Mav

[Earl Bonovich]

I am speculating that the "BOTH" is overriding the 28 day rule... I don't know if that is the case or not.

I haven't gotten a reply to my latest inquiry on it.

[Maverickster]

I am speculating that the "BOTH" is overriding the 28 day rule... I don't know if that is the case or not.

I haven't gotten a reply to my latest inquiry on it.

Thanks, please let us know when you hear. The point of my post, though, was that the "28 day rule" would have no application outside of "BOTH", so if BOTH is "overriding" it and causing the recording of First Runs/Repeats/Duplicates, then, sort of by definition, the "28 day rule" doesn't exist. The only possible "place" for the 28-day-rule is in BOTH to exclude Duplicates from the recording list.

--Mav

[Earl Bonovich]

Thanks, please let us know when you hear. The point of my post, though, was that the "28 day rule" would have no application outside of "BOTH", so if BOTH is "overriding" it and causing the recording of First Runs/Repeats/Duplicates, then, sort of by definition, the "28 day rule" doesn't exist. The only possible "place" for the 28-day-rule is in BOTH to exclude Duplicates from the recording list.

--Mav

You would have REPEAT option as well.... where the 28 day rule could still apply

[gcisko]

You are also not talking Red-Apples to Red-Apples... the guide data may have been different back when you were using your ultimate TV.


I figured you would point this out as I was writing my reply :)


I record several shows that are on multiple times a night, and replayed later in the week... and I only get single records
I do also have American Chopper setup from TLC, and I only get the single records, I do not get multiples.

Do you have them setup for First Run's only? or Both?
Most of my settings are First Run only...

I have American Gladiators from ECL set to BOTH, and that does record the second replay later in the evening.
So it is possible that the BOTH acts as not only recording repeats, but also records duplicates.

Yes I will have to check the series link more closely. I thought I had changed it to First Run. And maybe the problem was when I was recording from Discovery Theater (using TLC ever since it went HD). Thanks for the tips.

[gcisko]

I am speculating that the "BOTH" is overriding the 28 day rule... I don't know if that is the case or not.

I haven't gotten a reply to my latest inquiry on it.

Probably the same for a recording that is already on the hard drive??? That is my biggest issue with all this. Again I have to check to see if I have American Chopper set to BOTH. I sort of hope I do, so that would explain it.

[Maverickster]

You would have REPEAT option as well.... where the 28 day rule could still apply

I suppose that's true. Once you hear back from your contacts on how this is supposed to work, we'll have a better picture, but this is how I understood it was *supposed* to work (even though, in practice, it's far from perfect) for First Runs, Repeats, and Duplicates:

FIRST RUNS: Record only First Run Episodes (does not record Repeats or Duplicates on basis of First Run/Repeat distinction in Guide Data)

BOTH: Records First Runs and Repeats (does not record Duplicates on basis of "28 day rule" and "already on hard drive rule")

REPEATS: Records Repeats only (does not record First Runs on basis of First Run/Repeat distinction in Guide Data and does not record Duplicates on basis of "28 day rule" and "already on hard drive rule").

Is that your understanding as well?

[rudeney]

Thanks, please let us know when you hear. The point of my post, though, was that the "28 day rule" would have no application outside of "BOTH", so if BOTH is "overriding" it and causing the recording of First Runs/Repeats/Duplicates, then, sort of by definition, the "28 day rule" doesn't exist. The only possible "place" for the 28-day-rule is in BOTH to exclude Duplicates from the recording list.

Here’s how I understand the issue:

The guide data has a flag on every show indicating first-run or repeat.

The option for “first-run” vs. “both” is only available for series links and auto-records even though all shows in the guide have the first-run/repeat flag.

If your series link or auto-record is set for “both”, it ignores this flag. If a recording is set for “first run”, it will only record a show when the guide data indicates it is first-run (i.e. not a repeat).

The 28-day rule dictates that if you delete a show, it will not re-record it for the next 28 days, even if it meets your criteria. This would only be effective for series links or auto records, because single-show schedules are removed form the To-Do list once recorded.

For a series link or auto-record set for “first run”, it should only record if the guide data indicates that it is a first-run and not a repeat. If you delete it, assuming the guide data is correct, that show would never record again, regardless of the 28-day rule, because it will never again be a “first run”; any future broadcast of the show will by definition be a repeat.

For a series link or auto-record set for “both”, it will record regardless of the guide data’s first-run/repeat indicator. If you delete the show, it won’t record it again within the next 28 days.

My guess is that there are several logic flaws. First, I believe that the 28-day rule is only considering deleted recordings and not existing, saved recordings. Second, I believe that the “first-run” setting overrides the 28-day rule. Finally, I believe the guide data often improperly flags shows as “first-run”. All of this together makes the system record shows that we think it should not. I guess that’s somewhat better than the alternative, which is not recording a show that it should.

[Earl Bonovich]

If you want an IDEA on what the raw guide data looks like:
http://tmsdatadirect.com/docs/tv/

This is an older reference site (looks like things haven't been updated in a while)... but it will give you a fairly good idea what the raw data structure looks like for the GUIDE data.

[Maverickster]

Here’s how I understand the issue:

The guide data has a flag on every show indicating first-run or repeat.

The option for “first-run” vs. “both” is only available for series links and auto-records even though all shows in the guide have the first-run/repeat flag.

If your series link or auto-record is set for “both”, it ignores this flag. If a recording is set for “first run”, it will only record a show when the guide data indicates it is first-run (i.e. not a repeat).

The 28-day rule dictates that if you delete a show, it will not re-record it for the next 28 days, even if it meets your criteria. This would only be effective for series links or auto records, because single-show schedules are removed form the To-Do list once recorded.

For a series link or auto-record set for “first run”, it should only record if the guide data indicates that it is a first-run and not a repeat. If you delete it, assuming the guide data is correct, that show would never record again, regardless of the 28-day rule, because it will never again be a “first run”; any future broadcast of the show will by definition be a repeat.

For a series link or auto-record set for “both”, it will record regardless of the guide data’s first-run/repeat indicator. If you delete the show, it won’t record it again within the next 28 days.

My guess is that there are several logic flaws. First, I believe that the 28-day rule is only considering deleted recordings and not existing, saved recordings. Second, I believe that the “first-run” setting overrides the 28-day rule. Finally, I believe the guide data often improperly flags shows as “first-run”. All of this together makes the system record shows that we think it should not. I guess that’s somewhat better than the alternative, which is not recording a show that it should.

Right, sort of. Let's define some terms to make this easy (these are the meanings intended in my prior posts and are carry-over terms from TiVo, I believe):

"First Run" means the first airing of an Episode.

"Repeat" means any airing of an Episode that is not a First Run.

"Duplicate" means a Repeat that has been previously recorded.

"28 Day Rule" means a rule that the HR20 will not record a Duplicate which is either (a) currently in the Recordings List; or (b) was in the Recordings List in the immediately preceding 28 days.

Obviously, the 28 Day Rule has no bearing on First Runs. So, if you have "First Runs Only" selected, it's irrelevant. The only thing relevant if you have "First Runs Only" selected is the designation of the episode as a First Run or Repeat in the guid data.

The 28 Day Rule should apply, however, in both the "Both" selection and "Repeat" selection. If it does not, then selecting "Both" is effectively selecting to record First Runs, Repeats, AND Duplicates.

FWIW, what I'm driving at here is that there is no "28 day rule"; or, at the very least, if there is, it doesn't work properly.

--Mav

[rudeney]

If you want an IDEA on what the raw guide data looks like:
http://tmsdatadirect.com/docs/tv/

This is an older reference site (looks like things haven't been updated in a while)... but it will give you a fairly good idea what the raw data structure looks like for the GUIDE data.

Thanks, Earl. This is very interesting. There are actually two fields – “new” and “repeat” (see excerpt below). I wonder what happens if both are missing or if both are present and disagree?

<xsd:attribute name="repeat" type="xsd:boolean" use="optional">
- <xsd:annotation>
<xsd:documentation xml:lang="en">Designates a program which has been aired previously.</xsd:documentation>


<xsd:attribute name="new" type="xsd:boolean" use="optional">
- <xsd:annotation>
<xsd:documentation xml:lang="en">Designates a program which is new, not previously aired.</xsd:documentation>

[Craigbob]

TiVo has 10 years of manipulation to it's guide data processing to compensate for issues with the guide data... and it is not "perfect" in that manipulation..... There are times that even TiVo can't handle some of the guide data that it is given.

With the guide data it is GIGO Garbage In - Garbage out.

The issues with First Run/Repeat is HIGHLY dependent on the show.

So what show are you seeing an issue with?
What we have seen over the last two years, the "cable" networks are some of the worst with this issue..... not so much the network channels.

I have 41 Series Links, most of them are on Network TV... and I haven't had many issues with First Run vs Repeat (especialy over these last three months where there have been a TON of repeats). I get a random time it records a repeat, but upon investigating... it is typically an episode, that had generic or no detailed guide data to it, so the system was just falling back to "record just in case", let the user delete it if they don't want it.

Earl,

I have an HR20 and the show I have the most Problem with Star Trek: Enterprise on HD Net (79). It conflicts with episodes that get shown earlier in the day and episodes that are on Sci-Fi channel (244).


I've tried setting just for 1st run on 79 (Mon. @ 6:00pm) and wind up with shows from 10:00 am being recorded as well as shows on 244 being recorded. It is very frustrating to me. I wound up setting a manual SP for that time. But then I don't know what episode I'm getting since the info area is blank.

You wanted an example there it is.

Craig

[Maverickster]

Earl,

Any word from DirecTV on the "28 Day Rule"?

Thanks.

[Maverickster]

Earl,

Any update on this?

Thanks!

--Mav

[t_h]

I'm having a little problem with this as a new user, and I hope to god its not as bad as it seems and theres some explanation.

We hooked up the HR20 last night with its new external 750gb seagate in an MX-1, and let it do its thing overnight to gather guide data. I queued up about 20 VOD's this morning on it when those showed up.

I sat with it this evening and put in all of our old season passes from our old tivo. That went very smoothly and it sure is a lot faster than the tivo was. First I set the defaults to "first run only" and "keep all episodes".

After I entered all the season passes, I looked at the to-do list to see a bunch of shows set up to record, but a bunch of them (like 10) were all repeats when I looked at the guide data they had dates from 2000-2006 and "repeat" in the description.

Not new or weird shows, stuff like law and order, family guy, no reservations, etc.

I went and deleted the obvious repeats.

The old tivo would VERY occasionally record an old show with an old date and "repeat" in the description...but holy cow I hope this directv dvr after several years in of seasoning can do better than picking up repeats on first run series links about half the time.

Did I do something stupid, or does setting the default really not set it properly, or does it record the first available episode of every series link even if its a repeat and you said "first run only"?

I'm sorry if this has been covered already, I looked at the tivo to hd dvr survivors guide and did a search. The technology change is wearing us out a little bit.