 |
During my preparation for the Flash CS4 workshop that I’m doing I started to look at a new Flash video format called F4V. So what is the F4V format? Basically it is Adobe’s wrapper for H.264 video. So why do we need this wrapper for an open standard like H.264? Well it turns out the answer is to overcome some of the limitations of H.264 like the inability to embed cue points. I can imagine in the future that we may try to add alpha channel support which H.264 also doesn’t support. |
So this proprietary wrapping makes total sense but unfortunately the cue point functionality doesn’t really work. When encoding a video in Adobe Media Encoder I chose the F4V format and then added an event cue point just like I would with FLV. When I then imported the video into Flash and loaded it into the FLVPlayback component I even saw the event cue point in the component inspector. At this point I was really happy. But then I wrote the standard cue point event handler but the event never fired when it hit the cue point.
Rich Shupe saw one of my tweets about it and emailed a way to actually respond to the cue point. The solution was to respond to the XMP data event of the NetStream class. The code sample is listed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | package { import flash.display.Sprite; import flash.media.Video; import flash.net.NetConnection; import flash.net.NetStream; public class F4VCuePoints extends Sprite { private var _vid:Video = new Video(); public function F4VCuePoints():void { var nc:NetConnection = new NetConnection(); nc.connect(null); var ns:NetStream = new NetStream(nc); ns.client = this; addChild(_vid); _vid.width = 200; _vid.height = 140; _vid.attachNetStream(ns); ns.play("scaly.f4v"); } public function onMetaData(info:Object):void { } public function onXMPData(info:Object):void { var xmpXML:XML = new XML(info.data); // parse using Namespaces var xmpDM:Namespace = new Namespace("http://ns.adobe.com/xmp/1.0/DynamicMedia/"); var rdf:Namespace = new Namespace("http://www.w3.org/1999/02/22-rdf-syntax-ns#"); var cueFrameRateString:String = xmpXML..xmpDM::Tracks..rdf::Description.@xmpDM::frameRate; var cueFrameRate:Number = Number(cueFrameRateString.substr(1,cueFrameRateString.length)); trace("frameRate for DVA Ticks divisor:", cueFrameRate); var cuePointList:XMLList = xmpXML..xmpDM::markers.rdf::Seq.rdf::li; var len:int = cuePointList.length(); trace("number of cuePoints:", len); for (var i:int=0; i < len; i++) { var cueXML:XML = cuePointList[i]; trace("\tname:", cueXML.@xmpDM::name); trace("\ttype:", cueXML.@xmpDM::cuePointType); trace("\tstartTime:", cueXML.@xmpDM::startTime/cueFrameRate, "\n"); } } } } |
Big thanks to Rich for passing this along. It is ridiculous that we have to write that amount of code to respond to F4V cue points. I am hoping that this implementation is not quite finished and we are planning to make it easier in the future.
Update: I have confirmed from an internal source that if you create a navigation cue point on an F4V file, it does not add a keyframe into the video file at that point. Until this is fixed I would advice simply using regular H.264 files with AS cue points.
Lee
Thanks, Lee. By a coincidence, I discovered the same bug yesterday. I spent an aggravating couple of hours trying to get f4v cue points to work. Since the cue points showed up in the Component Inspector, I assumed there was something wrong with my code. Then, on a whim, I switched over to using an flv and my code worked fine. ARG!
Thanks for making this info public!
Thinking about this a little more, since there IS a way to read f4v cue points, doesn’t this just mean Adobe needs to update the FLVPlayback component?
Hi Lee,
the most interesting part is missing: How to extract useful data out of the XML data in the onXMPData function. It’s not that easy …
For example, the time codes in the XMP data are stored as DVA Ticks, not seconds (!).
You can find a code example here at the bottom (German):
http://www.video-flash.de/index/cuepoints-aus-f4v-videos-uber-xmp-auslesen/
Oh sorry, scrollbars did not seem to work for the code sample. Just saw the beginning of the code
Sorry just fixed the scrollbar thing. All the code should be visible now.
I have a question…
I haven’t tested this but I’m sure other people have. Can Flash still play the h264 format that is not wrapped with F4V metadata?
To me it makes that it can but you never know.
It’d be quite dumb if you can’t because Adobe most likely added the h264 format because h264 is popping up everywhere.
@Mikko Oh yeah of course. You would only use F4V if you wanted to add cue points. Otherwise just use a regular MP4 or MOV wrapper. Flash will always be able to play regular H.264 files.
Sure thing, Lee. The code above was just a quick sample, as you said. I was working on an event-driven follow-up until I had to return to a pressing deadline. I’ve started a modified FLVPlayback component, a suggestion recently echoed among colleagues, and will get back to you when I have a chance to finish it.
I discovered this issue a few month ago but I didn’t know the XMP trick.
Thanks !
A post about F4V and cue points : http://www.brooksandrus.com/blog/2008/12/07/f4v-does-not-support-cue-points/
Thanks for posting this information, Lee. We must have been on the same wavelength this weekend, because I was researching the same stuff for the Adobe Press Video for Flash book I’m in the middle of updating. While this code you’ve posted from Rich Shupe is most certainly useful, it doesn’t get around the fact that there’s no native event dispatched for these type of cue points _during playback_. Jeff Kamerer at Adobe shared with a hack that enables ActionScript cue points to work as navigation cue points–which could be used with the onXMPData parser you have here (i.e. just register the XMP cue point as an ActionScript cue point with the FLVPlayback instance, and then implement Jeff’s hack). If you’d like that code to post here, I’ll try to dig up the email Jeff sent me a few years ago.
Oh, and there’s also the issue of whether or not Adobe Media Encoder CS4 is generating actual keyframes at the cue point times–that’s the big advantage to use embedded cue points in the past with FLV. The cue point would always fire because keyframes are never dropped during playback. Can you find that out from the powers that be at Adobe, Lee?
@Robert Yes I’ll look into it. Is there a way to find out the precise location of keyframes in a video file?
How about doing a progressive download on videos encoded using h264 with another encode besides Adobe Media Encode?
I remember hearing at one point that mp4 should have metadata about the file size of the file in the beginning or some junk like that and because files encoded without the Adobe Media Encoder don’t preload nicely.
Sorry if these are totally ridiculous questions but I just haven’t had a chance to play around with h264 encoded videos in flash/
Thanks for the post.. I haven’t had to work with this format yet, but I am glad I have a heads whenever it pops up in the future.
@Mikko I use QT Pro to encode my files and they work perfectly.
@Lee – keyframes are available as part of the standard MPEG4-AVC onMetaData callback that gets fired by NetStream. There’s a “seekpoints” array returned as part of the metadata event that contains timestamps and their byte offsets.
Maybe I’m quibbling, but .f4v isn’t a “wrapper”. There’s nothing different between the bitstream format of a mp4, m4v or f4v file. XMP data is by definition non-destructive to the container bitstream and can be injected into an MPEG4 Part 14 container. So the only thing Adobe did was change the file extension, not the relevant bits.
Timed-Text tracks might be an area to explore for those seeking event notifications from MPEG4-AVC. They seem to be the way Apple delivers audio books (.m4a /.m4b files) that have a series of images synced to audio. Flash Player exposes the binary image data and text events via TT.
Hi Lee,
You were discussing the use of wrapper earlier, and is it correctly understood that the FLV format, doesnt support H.264 codec? In case you want to make use of it, you have to convert to F4V, MP4 or MOV, right?
Do you know Lee this problem was making me crazy
thanks for sharing
Lee,
Is what @Brooks said true? Are the files from the encoder merely renamed H264 movs and not a new file format?
@Mikko – You can use QuickTime Pro or Apple’s Compressor application to encode H.264. I’ve created a post with some specific settings for encoding with Compressor (or QT Pro for that matter) here: http://www.luunmedia.com/2008/12/07/encoding-h264-for-flash-using-quicktime-and-compressor/
Can you import a mp3 as audio when encoding to f4v in the flash media encoder all in one step?
Why would I want to parse the XMP data when I could pick another format that’s arguably more “standard” such as the XML format used in Captionate? That is, while this solution is helpful, it’s just reading in a separate xml source and parsing it. Personally, I’d fear that Adobe does something weird like changes the xmp spec on a whim. Maybe I’m missing something though. Thanks.
@Brooks Yes you are absolutely right. After looking into it more, we aren’t doing anything to modify the bits. But I would imagine that we might in future iterations. If we don’t then why have it at all?
@Tom You mean use a separate MP3 file to encode with a video? If so then no. You would need to combine them in your NLE first.
@Phillip Well the XMP data is a standard and we are apparently not changing it at all. But again, if we can’t embed real event and navigation cue points then I don’t see the value in this format yet.
The one thing I don’t understand about this .f4v format (like all other H.264 wrappers) is why there is such a long delay (8 seconds) prior to visually displaying the content on screen. The same file compressed under .flv would display in under 3. You can see this delay in this sample: http://www.lucid.it/nasa. This same file under the .flv (On2 VP6) loads almost immediately. I admit however, despite this delay bug, the quality the .f4v files are really incredible, especially with such incredibly low file sizes.
@Lee – I would be highly skeptical that Adobe will ever modify the bitstream format for their h.264 files. Doing so would essentially break compatibility with anyone else implementing the spec and if they do that there wasn’t much point in adopting h.264. However there are a number of reasons why Adobe, just as Apple did, decided to use custom file extensions. I’m not sure they fly, but I speculate about a few the the potential reasons in this post:
http://www.brooksandrus.com/blog/2008/12/07/f4v-is-retarded/
Warning: I don’t work for Adobe. Everything in my post is supposition, though based on good old fashioned deductive reasoning.
I think the f4v quality is impressive.
http://www.thirdfloorsolutions.com/fs/videoplaylist.html
Hi Lee,
I have a question.
you advice simply using regular H.264 files with AS cue points. I thaught that As Cue points are available only with the FLVPlayback. Is it true ?
i tried to make this code work correctly, but i did not get any result, i do not know what is going wronk,
i just created an AS(the code included inside the package provided) file with a blank fla file, i will be appreciated if you can help
as i spent big time to sort out this problem.
thanks in advanc
more about Cue Points in FLV and F4V
http://younsi.blogspot.com/2008/10/flv-cuepoints-in-flash-cs3-with-flash.html
Thanks mate, next time I’ll check your blog first. God damn what a mess XMP is! Great bytesArray workshop in Brighton in September btw
Hi Lee,
Why is the quality of my FLV or F4V different when played through Media Player Classic or VLC, than when I play it through my flash player on my website ? I have got the same dimensions as the FLV or F4V and tried Flash player 10 and 9.
Could you please enlighten me ?
Best regards,
Xander Smalbil
does anyone know if you can seek to navigation cue points with the flex videodisplay component? so far all i’ve seen is setting the playhead time to a specific time in seconds. nothing about going to a cue point.
Is this fixed : “I have confirmed from an internal source that if you create a navigation cue point on an F4V file, it does not add a keyframe into the video file at that point. Until this is fixed I would advice simply using regular H.264 files with AS cue points”.
Thanks for this post, helped me out of a bind. I’m interested to see what happens to the f4v format in the future, first time using it today, I’d been using flv and mp4 files, hope to use only one format now for a while though.
Hi Lee,
According to my tests not only does F4V not add keyframes at navigation cue point times and make it very hard to respond to cue points, but it also does not respect the “Set Key Frame Distance” option in Adobe Media Encoder. Even when this is set to 1, it seems to use a rate of 30, though the file size is increased.
I tested this by tracing the “seekpoints” property of the info object on the Metadata event for a variety of files – it always came out the same.
It seems to me this could be a problem for F4V as an viable option for the HTTP pseudostreaming now commonly performed. Can you confirm with the engineers that this is a bug in the format as it stands and give any indication of when Adobe might be able to fix it (in the Encoder, I guess)? Thanks!
I ran into this issue just yesterday. Glad I found this post or else I’d waste several more hours. I’m surprised this hasn’t been fixed already. The original post is dated almost a year ago. As Gavin asked, is there even a plan to have this fixed?
hey guys,
anybody of you guys know how to convert a F4V video clip in to a .wmv or .mov video file??
Can anybody at Abode confirm whether the problems with the F4V format, outlined above, have been solved with the CS5 release?
CS5 for Flash and Premiere is now out. Have they now fixed this issue with the lack of frame accurate cuepoints in the f4v format without need to this troublesome and imprecise workaround?
That’s awesome. I wrote a class to allow you to use cue points with any video codec (you gotta add them at runtime though they aren’t actually inside the video file)
here it is:
http://jpdevries.com/blog/2010/06/cued-h-264-cue-points-in-flash/
Hi Lee,
Was this issue ever fixed?