Simple Range Compressor

I made brief mention in the previous post about a little Windows tool that I wrote.  It's called Simple Range Compressor.  You can download it from here.  There's a good reason that the word 'simple' is in the title:

The purpose of this program is to perform dynamic range compression.  What this means is that it will attempt to make all sound play at the same volume.  In terms of PseudoTV, it will try to make each show / movie play at the same volume.  It will also keep those shows / movies playing at that volume no matter what is happening on screen.  Going from dialog to explosions shouldn't make any difference.

First the why
There are two main reasons to use this.  First is equalizing volume across shows.  If you play back a few shows in a row then you know that they aren't the same volume.  If you then switch to a movie, those are generally much different.  I hate messing with the volume control all of the time...this prevents that need.

The second reason is for those that don't want to wake their roommate / wife / baby.  You can watch a movie and not have to worry that it will suddenly get extremely load, or that you turn it down and can no longer hear the dialog.  They are the same volume.

Now for the how
Windows Vista and Windows 7 changed how they did audio quite a bit from previous versions.  They allow easy access to many things that used to be driver-dependent.  Simple Range Compressor is a very small program.  I took the easy, but effective, way out.  When the program starts, it notes the volume level that you have set.  It will then try to have all sounds play at that level.  It does this by actually adjusting the volume position itself.  If you click on the volume control you can see that it will slide up and down on its own.  This is also why you don't want the volume to be set to the maximum when using this: there is no room for changing the volume of the sound...explosions are already at that maximum, and it can't increase the volume in order for you to hear dialog.

Why is it simple
There are a few things that all dynamic range compression algorithms have to deal with...threshold, attack time,  and release time.  This program deals with those concepts as well.  Thing is, I just don't think that anyone cares.  If I set a threshold to -75db, does someone really care enough to change that to -70?  If I have an attack time of 20ms, will it make a big difference if it's 15ms instead?  I've set these things to what sounds best on my videos.  Since, after finding the values that I liked, I have no desire to change them anymore, I decided that I just wouldn't make them options at all.  Start the program, minimize it to put it in the system tray, and forget about it.


I hope people find some value in this.  I've been searching for a while for something to do this sort of thing, and I think this is a good solution.  I'm asking that if you try it and like it, please donate $1.00.  Not a lot, but I'm considering it an 'app' and that seems like a reasonable price.


Jason

Version 2.0 Released

Version 2.0 is finally out the door.  Here are some of the new features:

Channel Rules

This is the big feature for this release.  You're now able to customize every channel with a set of (currently) 12 different rules.  Here is a quick rundown:

Scheduling - Want to watch The Daily Show every night at 7:00?  Schedule your channel to play it.

Don't play this channel - Since some rules use other channels as modifiers (scheduling and interleaving), you may not want to actually play your channel that only has dozens of Battlestar Galactica episodes.  Hide it with this rule.

Don't play a show on this channel - I want to watch my episodes of Alias in order, but they keep showing up on my ABC channel.  Excluding them is only a rule away.

Force Random / Real-Time / Resume - While most people use Real-Time mode (as recommended), you can now have a channel be different from the rest.

Rename a channel - Don't like the default channel name?  Rename it to "Chick Stuff" to show that only your wife wants to watch the crap that plays on this channel*.

Interleaving - Are you one of those people that likes bumpers?  Then just create a channel that has only the bumpers and use this rule on your main channel to interleave those bumpers in between your shows.

Only play unwatched shows - If the channel contents change a lot, there's no reason to watch the same shows over and over.  Hide things you've watched from this channel with ease.

Only play watched shows - Don't want to spoil a movie by accidentally seeing part of the middle?  Hide shows that haven't been watched yet.

Play shows in order - Now you can make sure that when you're watching Top Chef, the next episode that comes on will actually be the next episode in the series.

Always Keep Paused - Don't want to miss an episode of The Big Bang Theory?  Make sure that if you're not watching it, it stays paused.  This rule goes great with the "Play shows in order" and "Force resume" rules!


* this is really the channel name I have for where my wife's crappy shows play.


Channel Sharing


I'm an old man, so my bed time is 10pm.  If I'm in the middle of a show at 9:15 and want to go from downstairs to upstairs, I just turn off the TV and turn on my system in my bedroom.  The channels are shared between the two so I can keep watching where I left off.  Huzzah!

Setting this up is a simple matter of pointing each computer to the same directory in the addon settings:

Background Channel Loading


As someone without patience, I want everything to start as quickly as possible.  Can I get rid of the loading time all together?  No...sorry.  I can cut it down, though.  As long as at least one channel is ready to go, that channel will be started.  Every other channel will be updated and added to the system as soon as they are ready.  A little notification will be given to show that channels are loading:

And another is shown when a new channel is available:

Other Changes


There are some other new things going on, although they may not be as obvious.  Here's a quick rundown:



Directory-type channels are supported now.
The EPG clock can be set to use a 12-hour or 24-hour clock.
Channel creation for first-time users has been significantly improved.
Time accuracy between restarts is essentially perfect now...or close enough for government work.
The EPG should function a bit quicker now.





I hope you enjoy the changes.  There are still a couple of bugs to be worked out (IceLibrary doesn't work, yet) and a couple of features, but this version feels pretty solid to me.


As a side note, I've created a little Windows program (Vista and Windows 7 only) called the "Simple Range Compressor" that tries to maintain the same volume between shows by performing some dynamic range compression using the Windows volume slider...wacky, but it works:

Download it here, discuss it here.

Channel Scheduling

There is an update to stable-pre in the repository.  This has the standard bug fixes as should be expected for every release.  It also fixes (and slightly modifies) the interleave rule.  It's now possible to select a minimum and maximum amount for the interleaving value.  This means that the number of shows between each inserted episode will be chosen randomly between the minimum and maximum value.  If you pick 3 and 5, for example, then there will be between 3 and 5 episodes of the normal channel followed by a single episode of the inserting channel.  After that, 3 to 5 shows of the normal channel and 1 of the inserted.

It also has at least the initial version of the scheduling.  I've decided to rename this to "Best-Effort" scheduling.

Best-Effort Channel Scheduling


Using this rule you do not schedule a movie or show, you schedule one channel to be played at a certain time during another channel.  This gives all sorts of flexibility since the scheduled channel can use rules of its own.

Why is it best-effort instead of just real "play this show exactly at this time" scheduling?  Because that way would be really damn hard, if not impossible.

When you watch a normal broadcast channel, the length of everything from the shows to commercials to the black spots in between are all timed so that the final length of an episode of the Simpsons will be 30 minutes.  I don't have that control.  In PseudoTV I give most of the control to XBMC itself to play a playlist that is created.  The best that I can do is insert an item into that playlist that should play (based on the length of each show before it) at about the time you want.

Of course just inserting a show blindly at about that time would result in really shitty timing.  If a movie is normally playing between 6 and 8 and you've scheduled something for 7, what would I do?  Either way the show will be an hour off.  Ideally what I would do is to rearrange the shows so that something ends at exactly 7 so that the new show can be inserted.  As I discovered, this is classic computer science problem called the subset sum problem.  Can any of the values in this subset be added together to get to zero.  This is effectively the problem that I faced.

First attempt

My first goal was to solve this problem.  In this case, I could setup a range that was valid: anywhere between 120 seconds before or after the time slot was good enough.  I hoped that this would make it easier.  I also decided to confine my search to only combinations of up to 4 values.  So I created an array that had the sum of all values added together:

{ [ index 1: 1021, index 2: 1426, total: 2447 ], [ index 1: 1021, index 3: 709, total: 1730 ], ... , [ index 499: 639, index 500: 3285, total: 3924 ] }

On top of that, I appended all combinations of 3 values, and then all combinations of 4 values.  It would be only a matter of taking the time that I wanted and seeing if swapping the shows from before the time to after would get me to the value that I wanted.

This took quite a bit of memory.  For only 10 shows I would have a total of 45 combinations of two,  120 combinations of three, and 330 combinations of four.  This means that for only 10 shows I would have 495 array entries.  Now, computers have a lot of memory.  In reality, these 495 entries would probably take up about 6KB worth of memory, not a significant amount by any means.  As the total number of entries increases, though, the total amount of memory grows extremely quickly.  Going from 10 to 20 entries raises the memory requirements to 88KB, 30 entries takes up 431KB.

Even this isn't a huge amount.  The problem is that, on occasion, the size of a playlist will get into the thousands.  How much memory?  I honestly have no idea.  I pushed my test program up to 300 entries and before it crashes it was taking up over 1GB worth of memory.  Obviously, this approach isn't feasible.

It's worth mentioning that, yes, it could be optimized.  I could sort the entries by time and remove duplicates.  Since I can't swap the same show more than once I could eliminate duplicate indexes.  Even so, this trades memory usage for processor usage.  While I don't mind taking up some time, a script inside of XBMC should not be killing your computer...other things need to take precedence, like playing video (for instance).

Second attempt (the best-effort method)


So I still need to schedule shows, but having a script solve the subset sum problem for thousands of elements probably won't work.  What I really need is some crappy method that will work, even if it isn't ideal.

I went back to the ridiculous array I created.  With some experimentation, I realized that since the number of elements in the subset (playlist) actually allowed for a reasonable switch of two items to get my time correct. Essentially, I found that a combination of 2 items worked great in most situations.  Excellent, I can create the combinations of 2 array and just look for the closest match out of that.

This simple solution solved most of the problems.  There were situations that still didn't work properly, though.  To resolve this, I just put my function inside of a nice little loop:


for loops in range(5):
    newtime = self.rearrangeShows(showindex, lasttime, channeldata, channelList)

    if newtime == lasttime or newtime < 120:

        break

    lasttime = newtime

This gives the system 5 chances at fixing the time, by switching up to 5 pairs of shows.  If the difference in the time between when I scheduled the show and when it will actually play is ever less than 120 seconds (plus or minus) then it will just say "good enough".

This method works pretty damn well, surprisingly.  It isn't optimal.  It does, though, minimize memory usage and attempts to get the optimal show placement out of as few processor cycles as possible.

Results

The outcome of this algorithm, though, is heavily dependent upon the channel contents itself.  If you're scheduling a TV show on a channel that plays only movies, then it may be difficult for it to place the show very well.  There would be fewer playlist entries and since the times for each entry is so high (2 hours or so, compared to 25 minutes of a TV show) then it may be difficult to place a show when you want it.  Can I improve this algorithm?  I'm sure it could be better.  Will I?  I'm not sure it's worth while, honestly.  I think any results I get would only be incrementally better, and since the function is only a minor part of the PseudoTV package (and will probably rarely get used as it is) I don't think it's worth my time.  Hopefully everyone will be happy with the current results, though.

If you do end up with large discrepancies between when you schedule a show and when it plays, please report it!  While I probably won't change the algorithm much, I certainly will fix any bugs that show up.

File Locks

New Feature


First a brief description of a new feature upcoming in PseudoTV.  I have 2 systems that run XBMC and PseudoTV in my house, one in my living room and one in my bedroom.  There have been several times that I've wanted to stop watching something in the living room and continue it upstairs.  That is now possible.  There is a new option to specify the directory to store PseudoTV playlists and settings.  If two systems share this directory, then they will both use the same set of channels and (with only minor differences) they will play them at the same times.

Implementation

This leads to the second part of this post: File locks over a network when using multiple platforms.  What if I turn on TV 1 and it starts loading the channels.  I then turn on TV 2 and it starts loading.  If TV 1 modifies any of the playlists, TV 2 won't necessarily know and things will get screwed up.  Who is allowed to extend the channel lists?  Who is allowed to write to the settings?  This is where file locks come into play.

Locking a file on a local hard drive so that other programs can't access it is generally possible, as long as you're writing for that platform specifically.  It can even be done on networked drives, again, presuming that the systems are the same and you target them specifically.  XBMC runs on multiple platforms so I can't presume anything of that sort.  So how can I make sure that two instances don't start writing to the same file at the same time?  How do I lock files?

The code is actually pretty simple...here are the steps:

1. Create a general lock file master list (I use FileLock.dat).
2. In order to lock a file, the first step is for a system to rename the master list to some random number filename (87385.lock).  This is effectively the atomic operation needed for locking files.
3. In order to verify that it succeeded, it makes sure it can open the file for reading.  If it can, then it now has control over the master file lock list.
4. Look in the list to see if the file you wish to lock is already there.  If it is, then it's locked.  If it isn't, then add it to the list.
5. Rename the list back to the original name.

That's it.  The file is locked.

Issues


If only it where that simple.  The problems arise when things don't work like they should.  How does the system know that it needs to create the lock list, and not that it doesn't just need to wait for another system to rename it back?  What happens if a system crashes and doesn't release a lock file, is that file permanently locked?

I'll start with the first question.  If any given system can't get access the master list for some amount of time (10 seconds) then it needs to just create the list.  Since, once a system has control over the list, the operations themselves take almost no time at all then the timeout value is very reasonable.  Is this perfect?  No.  It will not scale very well.  I would not recommend using this method for 10 machines at once since it may cause problems.  For a small scale, though?  Great.

What about the second issue, where a file can become permanently  locked?  This is resolved by adding a random value to the same line as the locked file in the list.  For example:

918374, MyLockedFile.txt

Every few seconds, the owner of that file will re-lock the file with a new random number.  Whoever wants access to MyLockedFile.txt will just watch the list and make sure that number actually changes.  If it doesn't change for some amount of time (10 seconds) then it assumes that the lock is dead, and it rewrites it.  If it does change then the lock is valid.

There's nothing terribly difficult about this method, but it took me a while to figure out...hopefully others will benefit from this.  The python code where I use this is here.

Works in Progress

I've mentioned in the past that the next version of PseudoTV will have some additional features.  I'm going to go through some of those, and then in the next week(s) you'll be able to try them on stable-pre.

Directory Channels


This feature has been requested several times in the past, but I hadn't really understood the reason for it.  People wanted to create channels based on a certain directory of media that wasn't scraped into XBMC.  Why?  For myself, the motivation came when I discovered HD-Trailers.NET Downloader.  My wife laughs at me for having this, but I really like having a channel that plays all of the newest trailers.  So I setup this program to download the newest ones every night.  Of course these won't scrape into XBMC so I need a way to play them...hence directory channels.

Background Channel Loading


One complaint that I've personally had about PseudoTV was that, at times, startup would take too long.  I'd have to sit and wait for each channel to update.  This isn't a problem if all of your media is connected to the computer running XBMC, but one of my machines is connected over dreaded WiFi.  Channel updates could take up to 15 seconds each...I'm not patient enough to wait for 20-something channels to update.  In comes background updating.

On startup, only channels that are ready to go will actually be loaded.  You may end up with as little as 1 channel actually enabled when PseudoTV starts playing.  After that, it will update and load the rest of the channels in the background, giving you a little message that pops up after each one is available.  No more waiting!  Combined with the previous version's ability to load up to 5 days of data in advance in the background, there should be virtually no updating of channels on startup and if updating is necessary it will be done without forcing delays.  Ah, multithreading, I'm a fan.

Channel Rules


You may have noticed in the above screenshot the addition of a new button: Channel Rules.  This will hopefully allow me to give users a lot more flexibility over how each channel is setup.  It also doesn't force it on them, so you can safely ignore its presence if you want.  Here's what the rule screen (currently) looks like:

It will have a bit of a different interface that I hope people find intuitive enough.  It will contain only a list.  Each channel will start with no rules.  When you select an empty rule to modify, you get this:

From there you can select the rule type and then just fill in the rule options.  The number of rules in a channel isn't limited, but it's possible to create a garbage channel if you're not careful.  For example, if the channel type is TV show, the show is Firefly, and you make a rule that says not to play the TV show Firefly.  That would be stupid.  Don't do that.

I don't have a final list of rules yet, but I'm working on it.  This is the big feature that will delay this release, so please don't throw things at me if it takes too long!  I'll make sure to update the PseudoTV thread with any stable-pre updates.

Worth mentioning


The video parser will now determine the length of MOV files.  Turns out that they are virtually the same format as MP4, I only needed to send them to that same parser and everything worked.  Huh, who knew.

Version 1.2.1

Version 1.2.1

A new version was officially released yesterday, version 1.2.1.  Given that the "incremental" version number changed, you know that this is a small release.  So true.  Let's see what's in it, though.

First off, two new skins were added.  Here is the Simplicity skin by Zepfan:

And here is the JX 720 skin by Steveb:

As far as bug fixes go, there are several that were handled in here.  Only 1 of them is even worth mentioning, though, and it only because of its simplicity and the difficulty I had in finding it.  For a certain user, the AVI parser was breaking on one of his files.  We went back and forth several times, each time I added additional debugging code.  I finally narrowed it down.

When reading a chunk of data, I had this code:

def read(self, thefile):
    data = thefile.read(4)
        

    try:
        self.size = struct.unpack('i', data)[0]
    except:
        self.size = 0

    # Putting an upper limit on the chunk size, in case the file is corrupt
    if self.size < 10000:
        self.chunk = thefile.read(self.size)
    else:
        self.chunk = ''

The point of this is that it will read in the data associated with the chunk.  Not a big deal...read the size itself, and then read the data.  I even limit the amount of data to read just in case the file is corrupt.  I missed something important, though: the size value is signed.  This means that in a file that is corrupt, I may read a negative number.  In Python, performing a read with a negative size will read in the entire file, causing problems.  So the fix was simple:

    if self.size > 0 and self.size < 10000:
        self.chunk = thefile.read(self.size)
    else:
        self.chunk = ''

Not a big deal.

Next time I'll discuss some of the things going into 1.3.0.

Video Formats

One of the things that PseudoTV relies in order to populate the EPG correctly is knowing the runtime of all of your videos.  For the most part, XBMC tells it these in the JSON calls.  Sometimes, though, it will return the oh-so-useful value of zero.  This has absolutely no meaning to me.  I can't put a video in the EPG with some unknown length.  So I have to try and determine how long the video is myself.  This is done by reading the video file directly and determining how long it is.  It took me quite a bit of research to figure out how to do this on the 4 file formats that I support (MKV, MP4, FLV, and AVI), so I figured I'd post something as a reference for anyone doing this sort of thing.

All of these video formats are composed of blocks.  In some cases, you need to read a block to figure out what the next block will be (and hence the amount of data to read).  In other cases, the block has a size field that tells you how big the block is.

Let's start with the easiest of the 4 file types:

MP4
MP4 Parser Python code


Each block in MP4 looks like this:

    Block size - 4 bytes.  This is the entire size of this block, including the standard fields (size, type, etc).
    Box type - 4 bytes.  A small string of 4 characters that tell you what type of box it is (header, video, audio, etc).

If the box size is 1, then this means there is another field here to get the real size:
    Real block size - 8 bytes.  This just allows the box size to be bigger than 4 bytes.

If the box type is 'uuid' then there is another field here:
    Real block type - 16 bytes.  Again, this is so the block type can be expanded if necessary.

    Block Data - x bytes, based on the block size.  Remember, the block size includes the stuff you've already read in.

To verify that you're reading an MP4 file, the first block type in the file should be 'ftyp'.

In MP4, the Block Data can actually contain other blocks.  You must understand the "block type" format in order to use this.  If you don't know what the block type is, then you can skip over it using the block size.  While determining the video duration, though, I generally don't care about the blocks or their sub-blocks.  What I'm looking for is the block type 'moov'.  This contain the movie information.

So keep going through blocks until you've found the 'moov' block.  After it has been found, look for the sub-block 'mvhd'.  In order to do this, just start your block search inside of the "Block Data" section of the 'moov' block.  After you've found 'mvhd', the block data format is:

    Standard block junk - Between 8 and 32 bytes (see above).
    Version - 1 byte.
    Flags - 3 bytes.  Don't care about these.

If the version is 1:
    Created - 8 bytes
    Modified - 8 bytes
    Scale - 4 bytes
    Duration - 8 bytes
otherwise:
    Created - 4 bytes
    Modified - 4 bytes
    Scale - 4 bytes
    Duration - 4 bytes

Now that you have the duration, you just need to use the scale value to figure out the length:
    Total Length - duration / scale

FLV
FLV Parser Python code


The FLV format is a bit different than the others in that there is no header to just grab the duration from.  What you need to do is grab the last video block and just see what the timestamp is on it.  So here we go:

The first 3 bytes of an FLV should read 'FLV'.  From there, jump to the end of the file.  You need to go backwards and find the last video block.  To do that, read the final 4 bytes of the file.  This tells you how big the previous block was.  So you go back that amount and read the header of the block:

    Tag type - 1 byte.  This tells what type of block this is.  A value of 9 means video (what we're looking for).
    Data size - 3 bytes.  The size of the block.  This includes the header data.
    Timestamp - 3 bytes.  The time stamp of the block in ms.
    Timestamp ext - 1 byte.  This is actually the upper 8 bits of the timestamp value.

First of all, you need a real single time stamp value.  So voltron it up between the timestamp and timestamp ext value (final timestamp= timestamp ext << 24 | timestamp).  So now that you've grabbed the block header you're looking for a tag type of 9, the final video block.  If you have found it, then the length of the video is:

    Total Length - final timestamp / 1000

If you haven't found the video block, then the 4 bytes before the block header will tell you how big the previous block was.  Skip backwards that amount and see if that's a video block.  Keep doing this until you find what you're looking for.

MKV
MKV Parser Python code


The Matroska Media Container is very flexible, but kind of a pain in the ass.  Here we go.

Each block starts with an EBML ID.  The length of the EBML ID depends on the first nibble.  Here's the craziness:

    Read the first byte.  If the most significant bit (MSB) is 1, then you have the proper ID.
    If the MSB is zero, read another byte onto the end of your ID.  Now check the next bit in the nibble you read up above.  If it's 1, then stop.  Otherwise read yet another byte and look at the next bit.  In summery, if the nibble is 8, the number of bytes in the ID is 1.  If it's 4 then the number of bytes is 2.  A nibble of 2 is 3 bytes, and 1 is 4 bytes.

Good times.  The first EBML ID of the entire file should be 0x1A45DFA3.  If you don't see that, then you don't have an MKV.

Next is the data size of the block.  This is pretty much the same thing as the ID, except that it's the first byte that denotes how long the data size is.  In this case, though, don't count in the 1 as part of the data size.  For example, if you see:

1000 0010

Then your data size is 2.  Since there is a 1 as the MSB, you only read 1 byte.  Also you mask it out so you're left with a value of 2.  Hopefully this makes sense...

Finally, the block has the data portion.  The data size field does not include the bytes for itself or the EBML ID, so just read data size bytes to get the block data.

So skip through blocks until you find a block with an EBML ID of 0x18538067.  Once you've found it, you need to look for the proper sub-block.  This part is similar to the embedded blocks in MP4...sub-blocks are found by looking into the block data itself.  Find the sub-block with the EBML ID of 0x1549A966.  This is the video header block.

Each field inside of the video header is actually contained in a sub-block of the header itself.  So search inside of the block data of the video header for another block with the ID of 0x2ad7b1.  This is the timecode value.  Just read the number of bytes in the data size field to get the timecode.  You also need the duration field, which is again in the video header with a block ID of 0x4489.  It will either be a float or a double float depending on the data size.

Whew, so now you have the timecode and duration of the video.  The video duration:
    Total Length - (duration * timecode) / 1,000,000,000

AVI
AVI Parser Python code

This is probably one of the more common video formats right now.  Also, one of the more annoying to parse.  There are 2 types of items in this file: chunks and lists.  A chunk is just some data, while a list is a collection of other chunks and lists.  Whether a block is a list or chunk is determined by a 4 byte string, similar to the block type in MP4.  This type is called the fourCC value.  If the fourCC value is RIFF or LIST then you have a list on your hands, otherwise it's a chunk.

Since a list is the first think you'll get in an AVI file, here's the format:
    Initial fourCC - 4 bytes.  This will be either RIFF or LIST for a list.
    Data size - 4 bytes.  The total size of all of the data inside of the list.
    FourCC - 4 bytes.  This is the real fourCC of the list.  In the case of the first list of the file, this is "AVI " (note the space at the end, to make it 4 bytes).
    Data - x bytes.  The amount is based on the "Data size" field.

The format for a chunk:
    FourCC - 4 bytes.  This is the same as above, but for a chunk this will NOT be RIFF or LIST.
    Data size - 4 bytes.  Again, the size of the data inside of the chunk.  This value does not include these chunk header bytes.
    Data - x bytes.  This is the size of the "Data size" value.

The first thing in the file is a list with the fourCC of "AVI ".  The data size of this list will just be the size of the file...not helpful.  So start reading inside of that all-encompassing list's block data.

Inside of the "AVI " list you should find another list with a fourCC of "hdrl".  Inside of the "hdrl" list is a chunk with the fourCC of "avih".  This is the header you're looking for.  Excellent.  Let's look at the format for the data inside of "avih":

    Micro-sec per frame - 4 bytes.
    Max bytes per sec - 4 bytes.
    Padding Granularity - 4 bytes.
    Flags - 4 bytes.
    Total frames - 4 bytes.
    Initial frame - 4 bytes.
    Number of streams - 4 bytes.
    Suggested buffer size - 4 bytes.
    Width - 4 bytes.
    Height - 4 bytes.

For our purposes, we don't care about most of this.  One thing we do care about is the number of streams.  So what needs to be done is to go through all of the streams until we find the video one.  So keep getting data from inside of "hdrl".  You'll get a list of type "strl".  This contains a description of all of the streams.  For each stream inside of the "strl" list, you'll first get a header chunk:

    Type - 4 bytes.
    Handler - 4 bytes.
    Flags - 4 bytes.
    Priority - 2 bytes.
    Language - 2 bytes.
    Initial Frame - 4 bytes.
    Scale - 4 bytes.
    Rate - 4 bytes.
    Start - 4 bytes.
    Length - 4 bytes.
    Suggested Buffer - 4 bytes.
    Quality - 4 bytes.
    Sample size - 4 bytes.

Again, don't care about this stuff for the most part.  After this stream header, there may or may not be some number of lists related to this stream...ignore them.  What you're looking for is a stream header where the Type field is "vids".  This is the video header.  Ah ha!  So take the video stream header.  The duration of the file is:

    Total Length - Length / (Rate / Scale)

Hopefully this has all been helpful to you!

* If you liked this post please click on one of the ads, or donate through paypal to help support the PseudoTV developer

Background threads and JSON calls

One of the important new features in PseudoTV version 1.2.0 was the use of a background thread to update your channels while you watch them. This meant that, in general, you don't have to wait for the channels to update every time you start PseudoTV...they will just always be up-to-date. Huzzah, right? Yes...kind of.

Since this thing was put in, I've had numerous complaints regarding freezing in the EPG, constant buffering, and general crappiness. Why, oh why do these issues suddenly exist? XMBC hates me, that's why.

Starting a new thread in Python (the scripting language used by XBMC) is easy:

class myThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        # Do stuff here

aThread = myThread()
myThread.start()

Excellent, now we have a thread. That's the easy part. It runs in the background, the main thread still gets called as normal, the world is a wonderful place. Under normal conditions, this thread can be preempted and so it won't block normal operation. Ah, but we run into an issue when we issue a JSON call from inside that thread:

xbmc.executeJSONRPC(command)

This makes a call to an XBMC library function. This function may take up to 15 or 20 seconds to complete. The kicker? This function blocks any Python code from running, even from a different thread. That means no user keypress callbacks, no actions if the user stops the video...no nothing. That doesn't make for a pleasant experience. So, what to do. Thankfully, XBMC provides a webserver that can process JSON calls, and guess what? Since Python isn't directly calling the library function, it won't block the code in other threads. So that's the answer! Kind of. Users don't HAVE to have the webserver turned on. It's off by default...meaning they have to specifically turn it on for it to work. Here's the pleasant function that's required:

def determineWebServer(self):
    if self.discoveredWebServer:
        return

    self.discoveredWebServer = True
    self.webPort = 8080
    self.webUsername = ''
    self.webPassword = ''
    fle = xbmc.translatePath("special://profile/guisettings.xml")

    try:
        xml = open(fle, "r")
    except:
        self.log("determineWebServer Unable to open the settings file", xbmc.LOGERROR)
        self.httpJSON = False
        return

    try:
        dom = parse(xml)
    except:
        self.log('determineWebServer Unable to parse settings file', xbmc.LOGERROR)
        self.httpJSON = False
        return

    xml.close()

    try:
        plname = dom.getElementsByTagName('webserver')
        self.httpJSON = (plname[0].childNodes[0].nodeValue.lower() == 'true')
        self.log('determineWebServer is ' + str(self.httpJSON))

    if self.httpJSON == True:
        plname = dom.getElementsByTagName('webserverport')
        self.webPort = int(plname[0].childNodes[0].nodeValue)
        self.log('determineWebServer port ' + str(self.webPort))
        plname = dom.getElementsByTagName('webserverusername')
        self.webUsername = plname[0].childNodes[0].nodeValue
        self.log('determineWebServer username ' + self.webUsername)
        plname = dom.getElementsByTagName('webserverpassword')
        self.webPassword = plname[0].childNodes[0].nodeValue
        self.log('determineWebServer password is ' + self.webPassword)
    except:
        return

This ridiculous function directly opens up the settings file that XBMC creates, checks to see if the user has enabled the webserver, and grabs the username, password, and port for it. So if this data is detected then that's the method that will be used for JSON calls. If it isn't (self.httpJSON is False) then the normal library call is made.

It's actually pretty amazing how much of the code is just a mix of ways to bypass certain limitations.

Also, please enable your webserver if you use PseudoTV! I don't care what port or what mess of username and password you have...just turn that sucker on.

So there ya go, the mess of the background thread and the insanity required to get it to work at all. I may find some better way to do this in the future, but for the moment this is what we have.

* If you liked this post please click on one of the ads, or donate through paypal to help support the PseudoTV developer

Stable-pre branch

When I'm developing new features or adding bug fixes, all of the work is done and maintained in the repository in the stable-pre branch.  This allows users that don't want to wait for an official release to test out the new versions.  It also lets me get some help debugging the system before I push it out and get a pile of complaints because I broke everything.

So what does the current stable-pre branch have?  This will be version 1.2.1, a small bug fix (mostly) version.  Here is a list of changes current up there:

Features:

• Addition of 30-skip.  This was a request that I thought was a great idea (from primetime34, and several others before him) that was easy to put in.  Now, presuming you don't have the info window or EPG up, using the left or right arrows will skip forward or backward 30 seconds.  A simple thing, but it's a helluva lot easier than skipping around in 10 minute steps.
• Removed the built-in Transparency skin so because the author (ronie) built it directly into the skin itself.  Check it out:

Bugs:

• There was this weird problem where exiting PseudoTV by using the escape button would end up crashing XBMC.  Here is a bit of detail that you can skip if you don't care.  XBMC has various ways for plugins to catch the many things that are happening in the system.  For example, I needed to make a subclass of the video player so that I could catch when the user stopped the video in...unexpected ways.  If they closed it from the OSD, or if they used an http remote, for instance.  So now I can actually snag those and exit PseudoTV properly.  When the user presses the escape key and chooses to exit, I need to do the same thing: stop the video.  When the user wants to exit, I immediately stop the video, stop all of the threads, and close the instance of the Overlay class (the primary class in PseudoTV).  The issue was that after this class closed and was deleted, XBMC would send a message to my Player subclass to say that the video stopped.  The player would then tell the Overlay instance that the user wants to exit...the Overlay instance that was just deleted.  Hence the crash.  So now I just have a flag to prevent the Player from sending this message in this case.
• There was a stupid bug where selecting the currently playing channel from the EPG would start the show from some random place.  It just dealt with the way I am storing the timestamps for playing videos and how I calculate where the show should play at.
• I hadn't caught an exception thrown if the XBMC webserver refused the connection for the JSON call.  Stupid of me.
• There was another uncaught exception when determining the file size of an MKV file.

So that's it for the moment.  I'm working on fixing a few more bugs, and there may or may not be another feature in the next release.  Next time I'll talk about the background threading and the pain in my ass that it's turning out to be...

Pseudo Introductions

Welcome to the PseudoTV blog.  This site will attempt to show the current progress, upcoming features, and questionable coding practices related to the PseudoTV addon for XBMC.

First off, this is the heart of what PseudoTV is:

It takes your existing media collection of movies and TV shows and creates a custom TV experience out of it.  You have separate channels with your own shows playing.  Select what you want to watch, or just let the channel play in the background.

I've spent quite a bit of time on this relatively simple idea to flesh it out and make it usable on the XBMC platform.  I will continue to offer updates and fixes for this addon as long as it's users want to use it.