Kdenlive   bug tracker Home page

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0002799KdenliveFile Loadingpublic2012-10-23 00:142013-01-06 14:24
Reporteryellow 
Assigned Toddennedy 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusfeedbackResolutionreopened 
Platform64 bitOSUbuntuOS Version12.04
Product VersionRecent git 
Target VersionFixed in Version 
Summary0002799: Canon / Nikon / GH3 h264AVC MOV's incorrect levels handling compared to ffplay.
DescriptionImporting Canon / Nikon / GH3 h264AVC MOV's ignores full range flag VUI Options metadata set 'on' in mov and mp4 containers by camera firmware / encoder. This leads to the well worn description of crushed shadows and blown highlights.

It was possible in earlier versions of kdenlive to override levels handling by using the force full luma in the clips properties but I think recent changes to MLT has perhaps nullified this or ffmpeg libswscale has changed. :-)

ffplay respects the h264 full range flag metadata and squeezes the full range luma levels encoded by the above cameras into the 16 - 235 range ready for conversion to RGB so contrast and levels are then correct for display, but kdenlive / MLT doesn't it would appear also the force full range in clip properties is ineffective.
Steps To ReproduceImport the 16-235_flagon.mp4 uploaded below.

Preview in kdenlive, notice solid black and white horizontal bars, no 16 & 235 text visible.

Play the same files with ffplay and see how they should appear.

Try to force full luma in kdenlive clip properties and it has no effect.

btw although the name of the file suggests luma levels 16 - 235, the encode is actually full luma range the name relates to the 16 & 235 text which should be visible in the preview.
TagsNo tags attached.
Build/Install Method3rd party package
Attached Files? file icon 16-235_flagon.mp4 [^] (38,263 bytes) 2012-10-23 00:14
? file icon 16-235_flagoff.mp4 [^] (38,263 bytes) 2012-10-30 22:42

- Relationships

-  Notes
(0008537)
j-b-m (administrator)
2012-10-23 23:40

In fact, it seems a to be a problem in the MLT format conversion. Luma full range works when using an yuv consumer but not when using an rgb consumer.

For example, with your clip, this works fine:
melt 16-235_flagon.mp4 -consumer sdl_preview

However, the following does not work:
melt 16-235_flagon.mp4 -consumer sdl_still

The avformat filter_avcolour_space filter has some code to handle the full_luma issue, but it seems like it does not work correctly in this case. I do not fully understanding this code, so forwarding it on MLT's mailing list...

(0008540)
ddennedy (developer)
2012-10-24 06:45

fixed in mlt git commit d925903
(0008543)
yellow (reporter)
2012-10-26 00:57

Dan, thanks for looking at the bug report so quickly, but there still seems to be problems, on importing the clip into todays kdenlive from sunabs git PPA:

1. The thumbnail in Project Tree is generated as solid black and white bars.

2. The thumbnail generated when adding to the timeline is solid black and white.

3. The 16 & 235 text only appears at playback and as long as no RGB color space Freior effects are added. Timeline thumbnails update after play through and show a solid black and white bar on first frame thumbnail and the 16 & 235 text on the last thumbnail of the clip as a result.

4. Scrubbing the timeline doesn't show 16 & 235, whether effects are on it or not.

5. Adding a Frei0r effect such as SOP/SAT with no adjustments, so should have no effect on clip levels, playback shows no text. Assume the YCC to RGB conversion for adding effects doesn't first squeeze levels 16 - 235 which should happen as it is signaled by the h264 fullrange flag being set on.

6. When play back finishes the project or clip monitor reverts to showing solid black and white horizontal bars. Histogram shows the same.

7. Force full luma range in clip properties squeezes yuvj420p levels so into 16 - 235, so RGB 255 becomes 235 & RGB 0 becomes RGB 16.

8. Extracting a frame from the Project Monitor when force full luma is on in clip properties, gives same as item 7, '255' text that should be RGB white(255) in RGB is 235 and '16' text that should be RGB 16 is RGB 30.

9. Setting a clip to force full luma and applying a RGB Frei0r effect like SOP/SAT makes solid bars as per item 5 but as levels have been squeezed as item 7 the white RGB (255) bar becomes 235 and RGB (0) bar becomes RGB (16)

Checking encoded output in Avisynth using kdenlive lossless ffv1 in a matroska, in each case:

1. 16-235_flagon.mp4 (yuvj420p due to full range levels flagged on) gets luma squeezed into 16 - 235 encoded to yuv420p 16 & 235 text shows, extracted frame in Avisynth shows correct RGB values, outcome is correct, but 'still' preview within kdenlive, re item 6 above, showed solid black and white bars, so judging color, brightness, contrast in still preview whilst applying effects to Canon, Nikon & Panasonic GH3 MOV's is inaccurate.

2. 16-235_flagon.mp4 + an effect working in RGB like the SOP/SAT encoded out ffv1 shows solid black and white Horizontal bars, no 16 & 235 text, appears the YCC to RGB conversion is done without respecting the full range flagged source, what should happen is MOV source luma squeezed into 16 - 235 first, then the RGB conversion done. This encode route failed.

3. 16-235_flagon.mp4 + Force Full Luma in clip properties produces correct output as long as no RGB based Frei0r effects are added. Appears Force Full Luma has no effect on encoded output, the original MOV yuvj420p file is still squeezed into 16 - 235 yuv420p. ffmpeg complaint about incompatible pixel formats.

4. 16-235_flagon.mp4 + Force Full Luma in clip properties + Frei0r SOP/SAT default settings produces incorrect output ie: solid black and white horizontal bars, no text. Extracted image shows bars as 0 RGB & 255 RGB. So backs up previous encoder item 2, color space conversion to RGB done without first squeezing luma. Leads to loss of shadow and highlight detail, crushed blacks, compressed highlights if any RGB Frei0r plugins are applied.

Sorry for the drawn out list above but something is not right with levels handling of these camera source files when effects are involved. :-(
(0008544)
yellow (reporter)
2012-10-26 01:03

Last line should read:

"Sorry for the drawn out list above but something is not right with levels handling of these camera source files, when sat on a still frame in the clip or project monitors or RGB Frei0r effects are applied. :-( "
(0008545)
ddennedy (developer)
2012-10-26 01:48

I do not believe sunab's repo at https://launchpad.net/~sunab/+archive/kdenlive-svn [^] does not include this commit yet. The package version says 0.8.3+git20121023.440b85dd where 440b85dd is the sha1sum of the HEAD commit for that snapshot, and the commit with the fix is 3 commits later than that. Sorry you were probably misled by the date on the package instead of the commit hash.
(0008546)
ddennedy (developer)
2012-10-26 01:50

Also, unlike before, now I am getting the color_range flag from avcodec to make it automatically choose full range in colorspace conversion.
(0008547)
ddennedy (developer)
2012-10-26 01:59

You can try this non-package nightly build made with the build script (uses a launch script): http://builds.meltytech.com/kdenlive/kdenlive-ubuntu11.10-x86_64-20121025.tar.bz2 [^]
(0008568)
yellow (reporter)
2012-10-30 23:06

Dan you were right the commit wasn't included in the build I was using, also good to hear the color_range option now actually does something.

There's just a slight problem with the handling of a video file with 'Force Full Luma' activated in the clip properties.

I've uploaded a full luma range h264 file with the VUI Option 'fullrange' flagged off in the mp4 container, it contains the identical h264 stream as the flagged on file. The unflagged mp4 file reflects certain camera models like Sony NEX5n, FS100 16 - 255 luma in an MTS container.

Adding the flagged off file to kdenlive, the thumbnail generated and the clip preview reflect the expected and show just black & white horizontal bars.

On setting the 'force full luma' on for this flagoff mp4 the clip monitor and luma waveform update to reflect the change, the 16 & 235 text appear, the project tree clip thumbnail doesn't update but that's being picky.

However on playing the clip in the clip monitor, the 16 & 235 text disappear and only black & white bars are seen. But if I then select another clip in the Project Tree and play that, then go back and select the flagof file, the 16 & 235 text reappear in the clip monitor until I play the clip again.

Timeline behavior is slightly different, on adding the flagoff file with 'force full luma' set on to the timeline the project monitor and luma waveform change to horizontal black and white bars.

If I then play the timeline clip still no 16 & 235 text is seen BUT the luma waveform monitor does reflect the change and shows gradation for the 16 & 235 text even though the project monitor shows just black & white bars. If I then change to another open application and then go back to kdenlive, the refresh sends the luma waveform back to showing just a 0 & 255 line, no representation of the grey 16 & 235 text.

Replaying the clip on the timeline the project monitor still shows black & white bars only BUT once again the luma waveform reflects the 16 & 235 text.

If I pause the clip at that point I still see black and white bars, no text but luma waveform reflects text, then swap to another app then back the luma waveform once again refreshes and just shows the 0 & 255 lines again.

If I let the clip on the timeline play out to the end, once again the luma waveform updates to reflect the 16 & 235 grey text, the project monitor stays black and white right up to the last frame, on the last frame at still, the clip updates and shows the 16 - 235 text in the project monitor until I press play again.

Sorry for the long drawn out description again.
(0008569)
yellow (reporter)
2012-10-30 23:53

Forgot encoder output:

1. Flagoff encodes out as full range luma as per source file, no luma squeeze because ffmpeg treats the flagged off file a yuv420p not yuvj420p and just passes levels through as is.

2. Flagoff file + force full luma encodes out as full range yuv420p as 1 but gets previewed in kdenlive as 16 - 235, well nearly there as above comments. Useful for maintaining full levels in an intermediate file but previewing for final delivery 16 - 235 in kdenlive.

3. Flagon file (which contains full range luma) + force full luma still encodes out as squeezed luma, as if force full luma was not set on ie: yuvj420p (JPEG levels due to flag set on in h264 stream), ffmpeg complains incompatible pixel format, and encodes out limited range yuv420p to fit the RGB levels to YCC levels conversion. So to get levels to pass through from native camera source ie: Canon, Nikon DSLR's or Pano GH3 still have to use a 3rd party tool to switch the full range flag VUI Option to off so file gets treated as yuv420p not JPEG levels yuvj420p.

Maybe the ffmpeg color_range option is not meant for this purpose, ie: to force the passing of source levels straight through regardless of flag state.

I know when we discussed this sometime ago you said that you had no time to investigate item 3 which is on my part a bit of a pedantic requirement but at least now h264 files flagged full range are getting interpreted 'correctly' as per ffplay thanks to your recent fix. Many thanks.
(0008574)
j-b-m (administrator)
2012-11-03 14:16

I can confirm that file that the force_full_luma flag does not work correctly in MLT.

Since Dan's patch, the flagon file now plays correctly in Kdenlive. However, the problem I mentionned in my previous comment is still here when we need to force the luma flag on an RGB consumer:

this works fine (displays the grey numbers):
melt 16-235_flagoff.mp4 -consumer sdl_preview

However, the following does not work (no numbers):
melt 16-235_flagoff.mp4 -consumer sdl_still

In case this can help solve the problem....
(0008575)
ddennedy (developer)
2012-11-03 19:26

For me 'melt 16-235_flagoff.mp4 -consumer sdl_preview' does not display the numbers. ffplay gives me the same result.

'melt 16-235_flagoff.mp4 -consumer sdl_still' is NOT supposed to show the numbers unless you set.force_full_luma=1 on the producer.

Luma range conversion only works when doing YUV->RGB conversion (and not vice versa). So, when the processing pipeline does not require any colorspace conversion, then the force_full_luma option is not used.

If you are seeing some change in behavior between some point prior to the regression and after my fix, it is because in the meantime some optimizations were made in mlt to reduce colorspace conversions when not needed.

Also, when you play the sdl_preview consumer it uses YUV mode, but when you pause the sdl_preview consumer, it switches to RGB mode. And the kdenlive scopes pull RGB images _after_ the embedded video preview updates, which explains some of the interesting behavior yellow observed.

If we force RGB conversions with

melt 16-235_flagoff.mp4 set.force_full_luma=1 -filter frei0r.R -consumer sdl_preview

or

melt 16-235_flagoff.mp4 set.force_full_luma=1 -consumer sdl mlt_image_format=rgb24

or

melt 16-235_flagoff.mp4 set.force_full_luma=1 -consumer sdl_preview play.mlt_image_format=rgb24

then it shows the numbers whether playing or paused. Maybe Kdenlive will want to always force the consumer to fetch RGB images? (If so, that will cause a regression against another bug complaining about needless color conversions slowing things down.) I suppose I could make a change where when I see full_luma in the VUI or as mlt property, then I force the avformat producer to output RGB.

For the encoding, there is nothing in the MLT avformat consumer that explicitly sets the avcodec color_range option for affecting the output VUI flag. You can probably fix that with the AVOption color_range applied as a property on the avformat consumer as "color_range=2"
(0008576)
ddennedy (developer)
2012-11-03 19:45
edited on: 2012-11-03 19:47

"I suppose I could make a change where when I see full_luma in the VUI or as mlt property, then I force the avformat producer to output RGB."

I have this change ready to commit if we agree it is sensible. I should add that the downstream MLT services can get what they need/requested regardless of this change as MLT does just-in-time image format and colorspace conversion.

(0008603)
yellow (reporter)
2012-11-10 22:45

j-b-m, will this resolve the problem, hopefully no one's waiting for my comment on this, I'm no dev so it's all a bit beyond me. :-)

Thanks again Dan for providing a solution for this.
(0008605)
ddennedy (developer)
2012-11-11 22:41

It turns out that this commit I mention to force an RGB conversion with full_luma has a bad interaction with the YADIF deinterlacer that I have not yet figured out. Also, because deinterlacers operate in YUV, I really ought to force the RGB conversion downstream of that because most other filters can or only operate in RGB. So, this is currently pending additional analysis and work.
(0008606)
ddennedy (developer)
2012-11-12 02:16

Resolved now in git commit c48ed74 where I found a way to defer the forced RGB conversion to apply the full luma range option after the deinterlace step.
(0009041)
j-b-m (administrator)
2013-01-06 03:56

@yellow:

Can you confirm that some camcorders like SONY NEX line need to have the full_luma flag forced to display correctly? I have a patch ready for Kdenlive that can apply the "full luma" flag depending on the camcorder model...
(0009044)
yellow (reporter)
2013-01-06 14:24

Yes, there are many cameras that encode luma into the 16 - 255 range including Sony NEX 5n, FS100 and Canon camcorders like HV20, 30 & 40.

If full luma is forced for display (RGB) the force would increase gamma reducing contrast slightly then at encode time if effects have been added which forces a conversion to RGB then 16-255 YCC to RGB to 16-235 YCC output should maintain appearance as if luma is forced.

A media player won't force luma on these so if someone was to edit in kdenlive see a lower contrast image and then encode to YCC without any conversion to RGB happening then I believe the 16-255 levels will pass tbrough, when viewed in a media player where no full luma will be forced then image will look more contrasty than kdenlives preview.

I have an example on my blog at www.blendervse.wordpress.com about loss of detail with these camera sources. Hopefull explains better.

So I think forcing luma should not be done for display thus emulating a typical 8bit media player, but any color space conversion to RGB for effects should force full luma, then luma levels falling outside of 16-235 range could be manipulated rather than crushed, allowing them to be pulled into the levels zone where they will be displayed in a typical media player under user contol in the grading process.

I can create some test files to try and work through this.

- Issue History
Date Modified Username Field Change
2012-10-23 00:14 yellow New Issue
2012-10-23 00:14 yellow File Added: 16-235_flagon.mp4
2012-10-23 23:40 j-b-m Note Added: 0008537
2012-10-23 23:40 j-b-m Assigned To => j-b-m
2012-10-23 23:40 j-b-m Status new => acknowledged
2012-10-24 06:45 ddennedy Assigned To j-b-m => ddennedy
2012-10-24 06:45 ddennedy Status acknowledged => assigned
2012-10-24 06:45 ddennedy Note Added: 0008540
2012-10-24 06:45 ddennedy Status assigned => resolved
2012-10-24 06:45 ddennedy Resolution open => fixed
2012-10-26 00:57 yellow Note Added: 0008543
2012-10-26 00:57 yellow Status resolved => feedback
2012-10-26 00:57 yellow Resolution fixed => reopened
2012-10-26 01:03 yellow Note Added: 0008544
2012-10-26 01:03 yellow Status feedback => assigned
2012-10-26 01:48 ddennedy Note Added: 0008545
2012-10-26 01:50 ddennedy Note Added: 0008546
2012-10-26 01:59 ddennedy Note Added: 0008547
2012-10-30 22:42 yellow File Added: 16-235_flagoff.mp4
2012-10-30 23:06 yellow Note Added: 0008568
2012-10-30 23:53 yellow Note Added: 0008569
2012-11-03 14:16 j-b-m Note Added: 0008574
2012-11-03 19:26 ddennedy Note Added: 0008575
2012-11-03 19:45 ddennedy Note Added: 0008576
2012-11-03 19:47 ddennedy Note Edited: 0008576 View Revisions
2012-11-10 22:45 yellow Note Added: 0008603
2012-11-11 22:41 ddennedy Note Added: 0008605
2012-11-12 02:16 ddennedy Note Added: 0008606
2012-11-12 02:16 ddennedy Status assigned => resolved
2012-11-12 02:16 ddennedy Resolution reopened => fixed
2013-01-06 03:56 j-b-m Note Added: 0009041
2013-01-06 14:24 yellow Note Added: 0009044
2013-01-06 14:24 yellow Status resolved => feedback
2013-01-06 14:24 yellow Resolution fixed => reopened


Copyright © 2000 - 2014 MantisBT Team
Powered by Mantis Bugtracker