Version 1.2.1

Version 1.2.1

A new version was officially released yesterday, version 1.2.1.  Given that the "incremental" version number changed, you know that this is a small release.  So true.  Let's see what's in it, though.

First off, two new skins were added.  Here is the Simplicity skin by Zepfan:

And here is the JX 720 skin by Steveb:

As far as bug fixes go, there are several that were handled in here.  Only 1 of them is even worth mentioning, though, and it only because of its simplicity and the difficulty I had in finding it.  For a certain user, the AVI parser was breaking on one of his files.  We went back and forth several times, each time I added additional debugging code.  I finally narrowed it down.

When reading a chunk of data, I had this code:

def read(self, thefile):
    data = thefile.read(4)
        

    try:
        self.size = struct.unpack('i', data)[0]
    except:
        self.size = 0

    # Putting an upper limit on the chunk size, in case the file is corrupt
    if self.size < 10000:
        self.chunk = thefile.read(self.size)
    else:
        self.chunk = ''

The point of this is that it will read in the data associated with the chunk.  Not a big deal...read the size itself, and then read the data.  I even limit the amount of data to read just in case the file is corrupt.  I missed something important, though: the size value is signed.  This means that in a file that is corrupt, I may read a negative number.  In Python, performing a read with a negative size will read in the entire file, causing problems.  So the fix was simple:

    if self.size > 0 and self.size < 10000:
        self.chunk = thefile.read(self.size)
    else:
        self.chunk = ''

Not a big deal.

Next time I'll discuss some of the things going into 1.3.0.

Video Formats

One of the things that PseudoTV relies in order to populate the EPG correctly is knowing the runtime of all of your videos.  For the most part, XBMC tells it these in the JSON calls.  Sometimes, though, it will return the oh-so-useful value of zero.  This has absolutely no meaning to me.  I can't put a video in the EPG with some unknown length.  So I have to try and determine how long the video is myself.  This is done by reading the video file directly and determining how long it is.  It took me quite a bit of research to figure out how to do this on the 4 file formats that I support (MKV, MP4, FLV, and AVI), so I figured I'd post something as a reference for anyone doing this sort of thing.

All of these video formats are composed of blocks.  In some cases, you need to read a block to figure out what the next block will be (and hence the amount of data to read).  In other cases, the block has a size field that tells you how big the block is.

Let's start with the easiest of the 4 file types:

MP4
MP4 Parser Python code


Each block in MP4 looks like this:

    Block size - 4 bytes.  This is the entire size of this block, including the standard fields (size, type, etc).
    Box type - 4 bytes.  A small string of 4 characters that tell you what type of box it is (header, video, audio, etc).

If the box size is 1, then this means there is another field here to get the real size:
    Real block size - 8 bytes.  This just allows the box size to be bigger than 4 bytes.

If the box type is 'uuid' then there is another field here:
    Real block type - 16 bytes.  Again, this is so the block type can be expanded if necessary.

    Block Data - x bytes, based on the block size.  Remember, the block size includes the stuff you've already read in.

To verify that you're reading an MP4 file, the first block type in the file should be 'ftyp'.

In MP4, the Block Data can actually contain other blocks.  You must understand the "block type" format in order to use this.  If you don't know what the block type is, then you can skip over it using the block size.  While determining the video duration, though, I generally don't care about the blocks or their sub-blocks.  What I'm looking for is the block type 'moov'.  This contain the movie information.

So keep going through blocks until you've found the 'moov' block.  After it has been found, look for the sub-block 'mvhd'.  In order to do this, just start your block search inside of the "Block Data" section of the 'moov' block.  After you've found 'mvhd', the block data format is:

    Standard block junk - Between 8 and 32 bytes (see above).
    Version - 1 byte.
    Flags - 3 bytes.  Don't care about these.

If the version is 1:
    Created - 8 bytes
    Modified - 8 bytes
    Scale - 4 bytes
    Duration - 8 bytes
otherwise:
    Created - 4 bytes
    Modified - 4 bytes
    Scale - 4 bytes
    Duration - 4 bytes

Now that you have the duration, you just need to use the scale value to figure out the length:
    Total Length - duration / scale

FLV
FLV Parser Python code


The FLV format is a bit different than the others in that there is no header to just grab the duration from.  What you need to do is grab the last video block and just see what the timestamp is on it.  So here we go:

The first 3 bytes of an FLV should read 'FLV'.  From there, jump to the end of the file.  You need to go backwards and find the last video block.  To do that, read the final 4 bytes of the file.  This tells you how big the previous block was.  So you go back that amount and read the header of the block:

    Tag type - 1 byte.  This tells what type of block this is.  A value of 9 means video (what we're looking for).
    Data size - 3 bytes.  The size of the block.  This includes the header data.
    Timestamp - 3 bytes.  The time stamp of the block in ms.
    Timestamp ext - 1 byte.  This is actually the upper 8 bits of the timestamp value.

First of all, you need a real single time stamp value.  So voltron it up between the timestamp and timestamp ext value (final timestamp= timestamp ext << 24 | timestamp).  So now that you've grabbed the block header you're looking for a tag type of 9, the final video block.  If you have found it, then the length of the video is:

    Total Length - final timestamp / 1000

If you haven't found the video block, then the 4 bytes before the block header will tell you how big the previous block was.  Skip backwards that amount and see if that's a video block.  Keep doing this until you find what you're looking for.

MKV
MKV Parser Python code


The Matroska Media Container is very flexible, but kind of a pain in the ass.  Here we go.

Each block starts with an EBML ID.  The length of the EBML ID depends on the first nibble.  Here's the craziness:

    Read the first byte.  If the most significant bit (MSB) is 1, then you have the proper ID.
    If the MSB is zero, read another byte onto the end of your ID.  Now check the next bit in the nibble you read up above.  If it's 1, then stop.  Otherwise read yet another byte and look at the next bit.  In summery, if the nibble is 8, the number of bytes in the ID is 1.  If it's 4 then the number of bytes is 2.  A nibble of 2 is 3 bytes, and 1 is 4 bytes.

Good times.  The first EBML ID of the entire file should be 0x1A45DFA3.  If you don't see that, then you don't have an MKV.

Next is the data size of the block.  This is pretty much the same thing as the ID, except that it's the first byte that denotes how long the data size is.  In this case, though, don't count in the 1 as part of the data size.  For example, if you see:

1000 0010

Then your data size is 2.  Since there is a 1 as the MSB, you only read 1 byte.  Also you mask it out so you're left with a value of 2.  Hopefully this makes sense...

Finally, the block has the data portion.  The data size field does not include the bytes for itself or the EBML ID, so just read data size bytes to get the block data.

So skip through blocks until you find a block with an EBML ID of 0x18538067.  Once you've found it, you need to look for the proper sub-block.  This part is similar to the embedded blocks in MP4...sub-blocks are found by looking into the block data itself.  Find the sub-block with the EBML ID of 0x1549A966.  This is the video header block.

Each field inside of the video header is actually contained in a sub-block of the header itself.  So search inside of the block data of the video header for another block with the ID of 0x2ad7b1.  This is the timecode value.  Just read the number of bytes in the data size field to get the timecode.  You also need the duration field, which is again in the video header with a block ID of 0x4489.  It will either be a float or a double float depending on the data size.

Whew, so now you have the timecode and duration of the video.  The video duration:
    Total Length - (duration * timecode) / 1,000,000,000

AVI
AVI Parser Python code

This is probably one of the more common video formats right now.  Also, one of the more annoying to parse.  There are 2 types of items in this file: chunks and lists.  A chunk is just some data, while a list is a collection of other chunks and lists.  Whether a block is a list or chunk is determined by a 4 byte string, similar to the block type in MP4.  This type is called the fourCC value.  If the fourCC value is RIFF or LIST then you have a list on your hands, otherwise it's a chunk.

Since a list is the first think you'll get in an AVI file, here's the format:
    Initial fourCC - 4 bytes.  This will be either RIFF or LIST for a list.
    Data size - 4 bytes.  The total size of all of the data inside of the list.
    FourCC - 4 bytes.  This is the real fourCC of the list.  In the case of the first list of the file, this is "AVI " (note the space at the end, to make it 4 bytes).
    Data - x bytes.  The amount is based on the "Data size" field.

The format for a chunk:
    FourCC - 4 bytes.  This is the same as above, but for a chunk this will NOT be RIFF or LIST.
    Data size - 4 bytes.  Again, the size of the data inside of the chunk.  This value does not include these chunk header bytes.
    Data - x bytes.  This is the size of the "Data size" value.

The first thing in the file is a list with the fourCC of "AVI ".  The data size of this list will just be the size of the file...not helpful.  So start reading inside of that all-encompassing list's block data.

Inside of the "AVI " list you should find another list with a fourCC of "hdrl".  Inside of the "hdrl" list is a chunk with the fourCC of "avih".  This is the header you're looking for.  Excellent.  Let's look at the format for the data inside of "avih":

    Micro-sec per frame - 4 bytes.
    Max bytes per sec - 4 bytes.
    Padding Granularity - 4 bytes.
    Flags - 4 bytes.
    Total frames - 4 bytes.
    Initial frame - 4 bytes.
    Number of streams - 4 bytes.
    Suggested buffer size - 4 bytes.
    Width - 4 bytes.
    Height - 4 bytes.

For our purposes, we don't care about most of this.  One thing we do care about is the number of streams.  So what needs to be done is to go through all of the streams until we find the video one.  So keep getting data from inside of "hdrl".  You'll get a list of type "strl".  This contains a description of all of the streams.  For each stream inside of the "strl" list, you'll first get a header chunk:

    Type - 4 bytes.
    Handler - 4 bytes.
    Flags - 4 bytes.
    Priority - 2 bytes.
    Language - 2 bytes.
    Initial Frame - 4 bytes.
    Scale - 4 bytes.
    Rate - 4 bytes.
    Start - 4 bytes.
    Length - 4 bytes.
    Suggested Buffer - 4 bytes.
    Quality - 4 bytes.
    Sample size - 4 bytes.

Again, don't care about this stuff for the most part.  After this stream header, there may or may not be some number of lists related to this stream...ignore them.  What you're looking for is a stream header where the Type field is "vids".  This is the video header.  Ah ha!  So take the video stream header.  The duration of the file is:

    Total Length - Length / (Rate / Scale)

Hopefully this has all been helpful to you!

* If you liked this post please click on one of the ads, or donate through paypal to help support the PseudoTV developer

Background threads and JSON calls

One of the important new features in PseudoTV version 1.2.0 was the use of a background thread to update your channels while you watch them. This meant that, in general, you don't have to wait for the channels to update every time you start PseudoTV...they will just always be up-to-date. Huzzah, right? Yes...kind of.

Since this thing was put in, I've had numerous complaints regarding freezing in the EPG, constant buffering, and general crappiness. Why, oh why do these issues suddenly exist? XMBC hates me, that's why.

Starting a new thread in Python (the scripting language used by XBMC) is easy:

class myThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        # Do stuff here

aThread = myThread()
myThread.start()

Excellent, now we have a thread. That's the easy part. It runs in the background, the main thread still gets called as normal, the world is a wonderful place. Under normal conditions, this thread can be preempted and so it won't block normal operation. Ah, but we run into an issue when we issue a JSON call from inside that thread:

xbmc.executeJSONRPC(command)

This makes a call to an XBMC library function. This function may take up to 15 or 20 seconds to complete. The kicker? This function blocks any Python code from running, even from a different thread. That means no user keypress callbacks, no actions if the user stops the video...no nothing. That doesn't make for a pleasant experience. So, what to do. Thankfully, XBMC provides a webserver that can process JSON calls, and guess what? Since Python isn't directly calling the library function, it won't block the code in other threads. So that's the answer! Kind of. Users don't HAVE to have the webserver turned on. It's off by default...meaning they have to specifically turn it on for it to work. Here's the pleasant function that's required:

def determineWebServer(self):
    if self.discoveredWebServer:
        return

    self.discoveredWebServer = True
    self.webPort = 8080
    self.webUsername = ''
    self.webPassword = ''
    fle = xbmc.translatePath("special://profile/guisettings.xml")

    try:
        xml = open(fle, "r")
    except:
        self.log("determineWebServer Unable to open the settings file", xbmc.LOGERROR)
        self.httpJSON = False
        return

    try:
        dom = parse(xml)
    except:
        self.log('determineWebServer Unable to parse settings file', xbmc.LOGERROR)
        self.httpJSON = False
        return

    xml.close()

    try:
        plname = dom.getElementsByTagName('webserver')
        self.httpJSON = (plname[0].childNodes[0].nodeValue.lower() == 'true')
        self.log('determineWebServer is ' + str(self.httpJSON))

    if self.httpJSON == True:
        plname = dom.getElementsByTagName('webserverport')
        self.webPort = int(plname[0].childNodes[0].nodeValue)
        self.log('determineWebServer port ' + str(self.webPort))
        plname = dom.getElementsByTagName('webserverusername')
        self.webUsername = plname[0].childNodes[0].nodeValue
        self.log('determineWebServer username ' + self.webUsername)
        plname = dom.getElementsByTagName('webserverpassword')
        self.webPassword = plname[0].childNodes[0].nodeValue
        self.log('determineWebServer password is ' + self.webPassword)
    except:
        return

This ridiculous function directly opens up the settings file that XBMC creates, checks to see if the user has enabled the webserver, and grabs the username, password, and port for it. So if this data is detected then that's the method that will be used for JSON calls. If it isn't (self.httpJSON is False) then the normal library call is made.

It's actually pretty amazing how much of the code is just a mix of ways to bypass certain limitations.

Also, please enable your webserver if you use PseudoTV! I don't care what port or what mess of username and password you have...just turn that sucker on.

So there ya go, the mess of the background thread and the insanity required to get it to work at all. I may find some better way to do this in the future, but for the moment this is what we have.

* If you liked this post please click on one of the ads, or donate through paypal to help support the PseudoTV developer

Stable-pre branch

When I'm developing new features or adding bug fixes, all of the work is done and maintained in the repository in the stable-pre branch.  This allows users that don't want to wait for an official release to test out the new versions.  It also lets me get some help debugging the system before I push it out and get a pile of complaints because I broke everything.

So what does the current stable-pre branch have?  This will be version 1.2.1, a small bug fix (mostly) version.  Here is a list of changes current up there:

Features:

• Addition of 30-skip.  This was a request that I thought was a great idea (from primetime34, and several others before him) that was easy to put in.  Now, presuming you don't have the info window or EPG up, using the left or right arrows will skip forward or backward 30 seconds.  A simple thing, but it's a helluva lot easier than skipping around in 10 minute steps.
• Removed the built-in Transparency skin so because the author (ronie) built it directly into the skin itself.  Check it out:

Bugs:

• There was this weird problem where exiting PseudoTV by using the escape button would end up crashing XBMC.  Here is a bit of detail that you can skip if you don't care.  XBMC has various ways for plugins to catch the many things that are happening in the system.  For example, I needed to make a subclass of the video player so that I could catch when the user stopped the video in...unexpected ways.  If they closed it from the OSD, or if they used an http remote, for instance.  So now I can actually snag those and exit PseudoTV properly.  When the user presses the escape key and chooses to exit, I need to do the same thing: stop the video.  When the user wants to exit, I immediately stop the video, stop all of the threads, and close the instance of the Overlay class (the primary class in PseudoTV).  The issue was that after this class closed and was deleted, XBMC would send a message to my Player subclass to say that the video stopped.  The player would then tell the Overlay instance that the user wants to exit...the Overlay instance that was just deleted.  Hence the crash.  So now I just have a flag to prevent the Player from sending this message in this case.
• There was a stupid bug where selecting the currently playing channel from the EPG would start the show from some random place.  It just dealt with the way I am storing the timestamps for playing videos and how I calculate where the show should play at.
• I hadn't caught an exception thrown if the XBMC webserver refused the connection for the JSON call.  Stupid of me.
• There was another uncaught exception when determining the file size of an MKV file.

So that's it for the moment.  I'm working on fixing a few more bugs, and there may or may not be another feature in the next release.  Next time I'll talk about the background threading and the pain in my ass that it's turning out to be...

Pseudo Introductions

Welcome to the PseudoTV blog.  This site will attempt to show the current progress, upcoming features, and questionable coding practices related to the PseudoTV addon for XBMC.

First off, this is the heart of what PseudoTV is:

It takes your existing media collection of movies and TV shows and creates a custom TV experience out of it.  You have separate channels with your own shows playing.  Select what you want to watch, or just let the channel play in the background.

I've spent quite a bit of time on this relatively simple idea to flesh it out and make it usable on the XBMC platform.  I will continue to offer updates and fixes for this addon as long as it's users want to use it.