diff --git a/README.md b/README.md index 0e8ff8d..6fa53fa 100644 --- a/README.md +++ b/README.md @@ -204,7 +204,8 @@ register the key, the Event Log says so; choose another, such as `Ctrl-Shift-S - A recording that stops by itself keeps what it has, and so does quitting while recording. - When the screen is lost for a moment (a UAC prompt, the lock screen, a resolution change), the recording carries on once it is back. - - A recording a crash cut short is offered at the next start: into the library, or deleted. + - A recording a crash cut short is offered at the next start: into the library, or deleted. It + keeps all but its last two seconds or so. - **Settings:** `[video]` in the settings file: `fps` (30), `cursor` (true), and `capture`: `auto` (Desktop Duplication, else Windows.Graphics.Capture), `dxgi` or `wgc`. - **PrintScreen while recording** takes the whole screen under the cursor at once, without freezing diff --git a/check_targets.jai b/check_targets.jai index e33ffd2..20318ec 100644 --- a/check_targets.jai +++ b/check_targets.jai @@ -29,7 +29,7 @@ APP_NAME :: "mcap"; APP_FOLDER_NAME :: "mcap"; GIT_BRANCH :: "check"; GIT_HASH :: "-"; -VERSION :: "0.1.6"; +VERSION :: "0.1.7"; BUILD_DATE :: "-"; DEBUG :: true; COMPILE_TESTS :: true; diff --git a/first.jai b/first.jai index bb4b86d..0e75081 100644 --- a/first.jai +++ b/first.jai @@ -8,7 +8,7 @@ APP_NAME :: "mcap"; APP_FOLDER_NAME :: "mcap"; APP_BUNDLE_IDENTIFIER :: "com.example.mcap"; -VERSION :: "0.1.6"; +VERSION :: "0.1.7"; // Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated // builds never write over the ones you run by hand. build_directory := "bin"; diff --git a/modules/Capture_File/README.md b/modules/Capture_File/README.md index 1e21207..6e7f165 100644 --- a/modules/Capture_File/README.md +++ b/modules/Capture_File/README.md @@ -120,8 +120,10 @@ Format: "PNG" ## Screen recordings -A recording made with the mcap app's Ctrl+PrintScreen is a video: its original is the H.264 MP4 as -recorded, and its rendered image (kind 2) a frame from it as a PNG, as for an imported video. Its +A recording made with the mcap app's Ctrl+PrintScreen is a video: its original is the H.264 MP4, and +its rendered image (kind 2) a frame from it as a PNG, as for an imported video. It is recorded as a +fragmented MP4, so a crash still leaves one that plays, and stored as a regular one, its frames copied +as they are: one header first, with its length and the index of every frame. Its metadata.json is a still's. Its capture.json has a still's keys, with Format "MP4", and these: ``` diff --git a/modules/Screen_Record/encoder.jai b/modules/Screen_Record/encoder.jai index 3d37dee..cf570e4 100644 --- a/modules/Screen_Record/encoder.jai +++ b/modules/Screen_Record/encoder.jai @@ -4,10 +4,13 @@ // BGRA as it is and converts it on the GPU; the others are given NV12 or YUV 4:2:0, converted with // swscale (BT.709, limited range, as tagged). // -// The file is written fragmented (a fragment each keyframe, every 2 s), so one cut short by a crash -// still plays up to its last fragment. +// The file is written fragmented, a fragment at each keyframe, so one cut short by a crash still plays +// up to its last fragment. Frames are written only when the screen changes, so a count of frames can +// span any time: a keyframe is made every RECORD_KEYFRAME_SECONDS of time instead, which bounds what a +// crash loses. remux.jai makes the file a regular MP4 once it is done. RECORD_ENCODERS :: string.["h264_nvenc", "h264_amf", "h264_qsv", "h264_mf", "libx264"]; +RECORD_KEYFRAME_SECONDS :: 2; Encoder :: struct { format: *AVFormatContext; @@ -20,6 +23,7 @@ Encoder :: struct { input: AVPixelFormat; // what it is given: BGR0 as captured, or converted to NV12 or YUV420P width, height: s32; last_pts: s64 = -1; + last_key_pts: s64 = -1; // the last keyframe asked for header_written: bool; bytes: s64; // of packets written } @@ -95,6 +99,10 @@ encoder_write :: (e: *Encoder, pixels: *u8, pts: s64) -> ok: bool, error: string sws_scale(e.sws, *source, *stride, 0, e.height, frame.data.data, frame.linesize.data); } frame.pts = pts_out; + if e.last_key_pts < 0 || pts_out - e.last_key_pts >= RECORD_KEYFRAME_SECONDS * RECORD_TIME_BASE { + frame.pict_type = .I; // an IDR, with forced-idr: a new fragment + e.last_key_pts = pts_out; + } result := avcodec_send_frame(e.codec, frame); av_frame_unref(frame); @@ -147,11 +155,11 @@ encoder_try :: (e: *Encoder, name: string, width: s32, height: s32, fps: s32) -> options: *AVDictionary; set :: (key: string, value: string) #expand { av_dict_set(*`options, temp_c_string(key), temp_c_string(value), 0); } if name == { - case "h264_nvenc"; set("preset", "p4"); set("rc", "vbr"); set("cq", "23"); c.bit_rate = 0; + case "h264_nvenc"; set("preset", "p4"); set("rc", "vbr"); set("cq", "23"); set("forced-idr", "1"); c.bit_rate = 0; case "h264_amf"; set("rc", "cqp"); set("qp_i", "22"); set("qp_p", "24"); set("quality", "balanced"); case "h264_qsv"; set("global_quality", "23"); set("preset", "medium"); case "h264_mf"; set("rate_control", "quality"); set("quality", "70"); set("scenario", "display_remoting"); - case "libx264"; set("preset", "veryfast"); set("crf", "23"); + case "libx264"; set("preset", "veryfast"); set("crf", "23"); set("forced-idr", "1"); } result := avcodec_open2(c, codec, *options); av_dict_free(*options); @@ -205,6 +213,8 @@ encoder_free :: (e: *Encoder) { e.sws, e.format, e.stream = null, null, null; } +#scope_module + // What FFmpeg says an error code is, after what failed (temp). av_error :: (what: string, code: s32) -> string { buffer: [256] u8; diff --git a/modules/Screen_Record/module.jai b/modules/Screen_Record/module.jai index 8758186..2f6e209 100644 --- a/modules/Screen_Record/module.jai +++ b/modules/Screen_Record/module.jai @@ -529,6 +529,7 @@ rect_inside :: (inner: RECT, outer: RECT) -> bool { #load "dxgi_source.jai"; #load "wgc_source.jai"; #load "encoder.jai"; +#load "remux.jai"; #load "cursor.jai"; #import "Basic"; diff --git a/modules/Screen_Record/remux.jai b/modules/Screen_Record/remux.jai new file mode 100644 index 0000000..7885554 --- /dev/null +++ b/modules/Screen_Record/remux.jai @@ -0,0 +1,65 @@ +// A recording made into a regular MP4. While recording, the MP4 is written fragmented (encoder.jai), so +// a crash still leaves one that plays; but a fragmented MP4's header says nothing of its length or +// where its frames are, and one cut short by a crash never gets the index written at the end of a +// clean stop, so players show its time and seek in it badly. record_remux copies its packets, as they +// are, into a regular MP4: one header, with the length and the index of every frame, at the front +// (faststart), so it plays, seeks and streams as any MP4 does. Nothing is encoded again. + +// Copies source's video and audio packets into a regular MP4 at destination. packets: how many. +record_remux :: (source: string, destination: string) -> ok: bool, error: string, packets: s64 { + av_log_set_level(AV_LOG_ERROR); + input: *AVFormatContext; + result := avformat_open_input(*input, temp_c_string(source), null, null); + if result < 0 return false, av_error("FFmpeg cannot open the recording", result), 0; + defer avformat_close_input(*input); + result = avformat_find_stream_info(input, null); + if result < 0 return false, av_error("FFmpeg cannot read the recording's streams", result), 0; + + output: *AVFormatContext; + result = avformat_alloc_output_context2(*output, null, "mp4", temp_c_string(destination)); + if result < 0 || !output return false, av_error("FFmpeg cannot make an MP4", result), 0; + defer { + if output.pb avio_closep(*output.pb); + avformat_free_context(output); + } + + // Each input stream's output stream, or -1 for one left out (neither video nor audio). + map := NewArray(input.nb_streams, s32,, temp); + for 0..input.nb_streams - 1 { + in := input.streams[it]; + map[it] = -1; + if in.codecpar.codec_type != .VIDEO && in.codecpar.codec_type != .AUDIO continue; + out := avformat_new_stream(output, null); + if !out return false, "FFmpeg cannot add a stream", 0; + avcodec_parameters_copy(out.codecpar, in.codecpar); + out.codecpar.codec_tag = 0; // the MP4 muxer picks its own + out.time_base = in.time_base; + map[it] = out.index; + } + + result = avio_open(*output.pb, temp_c_string(destination), AVIO_FLAG_WRITE); + if result < 0 return false, av_error(tprint("FFmpeg cannot write %", destination), result), 0; + options: *AVDictionary; + av_dict_set(*options, "movflags", "+faststart", 0); // the header before the frames: a second pass at the end + result = avformat_write_header(output, *options); + av_dict_free(*options); + if result < 0 return false, av_error("FFmpeg cannot start the MP4", result), 0; + + packet := av_packet_alloc(); + defer av_packet_free(*packet); + packets: s64; + while av_read_frame(input, packet) >= 0 { // to the end, or to where a crash cut it + index := map[packet.stream_index]; + if index < 0 { av_packet_unref(packet); continue; } + av_packet_rescale_ts(packet, input.streams[packet.stream_index].time_base, output.streams[index].time_base); + packet.stream_index = index; + packet.pos = -1; + result = av_interleaved_write_frame(output, packet); // takes the packet's data + if result < 0 return false, av_error("FFmpeg cannot write to the MP4", result), packets; + packets += 1; + } + if !packets return false, "the recording holds no frames", 0; + result = av_write_trailer(output); + if result < 0 return false, av_error("FFmpeg cannot finish the MP4", result), packets; + return true, "", packets; +} diff --git a/src/app/screen_record.jai b/src/app/screen_record.jai index ea5b91a..a3ced58 100644 --- a/src/app/screen_record.jai +++ b/src/app/screen_record.jai @@ -285,21 +285,34 @@ screen_record_stop :: (shutdown := false) { // The recording's .mcap in the library, from the job's MP4, which is deleted once it is in: its path, // or why it could not be (temp). On the job's thread. +// +// The MP4 was written fragmented, so a crash leaves one that plays; it is made a regular MP4 first +// (record_remux), with its length and index in its header, as players need to show its time and seek +// in it. Should that fail, the MP4 goes in as it was. screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string { - _, size, found := file_modtime_and_size(job.recording); - if !found || size <= 0 return "", tprint("% is missing", job.recording); - image, video, decoded, decode_error := video_poster(job.recording); + _, recorded_size, found := file_modtime_and_size(job.recording); + if !found || recorded_size <= 0 return "", tprint("% is missing", job.recording); + mp4 := job.recording; + remuxed := tprint("%.regular.mp4", path_strip_extension(job.recording)); + remux_ok, remux_error, packets := record_remux(job.recording, remuxed); + if remux_ok mp4 = remuxed; + else { log_error("[record] % stays fragmented: %", job.recording, remux_error); file_delete(remuxed); } + defer if remux_ok file_delete(remuxed); + _, size, _ := file_modtime_and_size(mp4); + + image, video, decoded, decode_error := video_poster(mp4); if !decoded return "", tprint("the recording cannot be read back: %", decode_error); defer image_free(*image); preview, preview_width, preview_height, preview_ok, preview_error := image_preview_png(image, MCAP_PREVIEW_MAX_SIDE); if !preview_ok return "", preview_error; poster, _, _, poster_ok, poster_error := image_preview_png(image, max(image.width, image.height)); // at its own size if !poster_ok return "", poster_error; - hash, hashed := import_hash_file(.{path = job.recording, offset = 0, size = size}); - if !hashed return "", tprint("% cannot be read", job.recording); + hash, hashed := import_hash_file(.{path = mp4, offset = 0, size = size}); + if !hashed return "", tprint("% cannot be read", mp4); shot := job.shot; - if !shot.width { shot.width, shot.height = image.width, image.height; } // a crash's leftover: no more is known + known := shot.width > 0; // a crash's leftover knows no region, window or encoder + if !known { shot.width, shot.height = image.width, image.height; } doc := New(Capture_Document,, temp); capture_document_init(doc, temp); doc.comment = tprint("Recorded by % %.", APP_NAME, VERSION); @@ -308,9 +321,11 @@ screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string import_add(doc, metadata, "CaptureDate", job.date); import_add(doc, metadata, "WindowName", import_text(shot.window_title)); info := capture_add_file(doc, "capture.json"); - import_add(doc, info, "Region", ifx shot.whole_screen then "screen" else "selection"); - import_add_number(doc, info, "Left", xx shot.left); - import_add_number(doc, info, "Top", xx shot.top); + if known { + import_add(doc, info, "Region", ifx shot.whole_screen then "screen" else "selection"); + import_add_number(doc, info, "Left", xx shot.left); + import_add_number(doc, info, "Top", xx shot.top); + } import_add_number(doc, info, "Width", xx shot.width); import_add_number(doc, info, "Height", xx shot.height); if shot.app_path import_add(doc, info, "AppPath", import_text(shot.app_path)); @@ -337,7 +352,7 @@ screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string m.rendered = poster; path, folder_error := screen_capture_out_path(name); if folder_error return "", folder_error; - written, write_error := mcap_write_streamed(path, m, .{path = job.recording, offset = 0, size = size}); + written, write_error := mcap_write_streamed(path, m, .{path = mp4, offset = 0, size = size}); if !written return "", write_error; file_delete(job.recording); return path, ""; diff --git a/src/test/import_tests.jai b/src/test/import_tests.jai index e108f68..a482cef 100644 --- a/src/test/import_tests.jai +++ b/src/test/import_tests.jai @@ -732,7 +732,10 @@ screen_record_tests :: () { m, parsed := mcap_parse(cast([] u8) read_entire_file(path,, temp)); expect(parsed, "it reads back"); if parsed { - expect(m.original.count == mp4_size && ends_with(m.original_name, ".mp4"), "the MP4 is its original: % of % bytes, %", m.original.count, mp4_size, m.original_name); + // Recorded fragmented, it goes in a regular MP4: the same frames, its header first. + original := cast(string) m.original; + expect(ends_with(m.original_name, ".mp4") && slice(original, 4, 4) == "ftyp" && find_index_from_left(original, "moof") < 0 && find_index_from_left(original, "moov") < find_index_from_left(original, "mdat"), + "the MP4, made regular, is its original: % (% bytes, % recorded)", m.original_name, m.original.count, mp4_size); expect(m.rendered_kind == .OWN, "a frame of it is its rendered image"); poster, decoded := image_decode(m.rendered); expect(decoded && poster.width == WIDTH && poster.height == HEIGHT, "at the recording's size"); @@ -779,6 +782,36 @@ screen_record_tests :: () { } file_delete(leftover_path); } + + // One a crash cut short: a frame every 100 ms for 5 s makes fragments at 0, 2 and 4 s (a + // keyframe each 2 s, however few the frames); cut at the last, as a crash would, it goes into the + // library as a regular MP4 (header first, no fragments) of the 4 s before the cut. + encoder = .{}; + encoder_open(*encoder, mp4, WIDTH, HEIGHT, 30, ""); + for frame: 0..49 encoder_write(*encoder, pixels.data, frame * 9000); + encoder_close(*encoder); + whole := read_entire_file(mp4,, temp); + last_fragment := -1; + for 4..whole.count - 4 if whole[it] == #char "m" && slice(whole, it, 4) == "moof" last_fragment = it - 4; + expect(last_fragment > 0, "the recording is fragmented"); + write_entire_file(mp4, slice(whole, 0, last_fragment)); + crashed: Screen_Capture_Job; + crashed.video = true; + crashed.recording = mp4; + crashed.date = "2026-10-10 07:48:52"; + crashed_path, crashed_error := screen_record_write(*crashed); + expect(!crashed_error, "a cut recording is written: %", crashed_error); + if !crashed_error { + m3, parsed3 := mcap_parse(cast([] u8) read_entire_file(crashed_path,, temp)); + original := cast(string) m3.original; + moov := find_index_from_left(original, "moov"); + mdat := find_index_from_left(original, "mdat"); + expect(parsed3 && moov > 0 && moov < mdat && find_index_from_left(original, "moof") < 0, "a regular MP4, its header first"); + crashed_entry := catalog_load(crashed_path); + expect(crashed_entry && crashed_entry.duration > 3.5 && crashed_entry.duration < 4.5, "the 4 s before the cut: %", ifx crashed_entry then crashed_entry.duration else -1); + if crashed_entry catalog_entry_free(crashed_entry); + file_delete(crashed_path); + } } } diff --git a/tools/record_probe.jai b/tools/record_probe.jai index 06b6a17..1368694 100644 --- a/tools/record_probe.jai +++ b/tools/record_probe.jai @@ -20,6 +20,7 @@ // WDA_EXCLUDEFROMCAPTURE: is it out of the recording, and the other in? // -animate moves a square across the region the whole time, so the screen changes every // frame: for measuring throughput. +// -remux IN OUT a recording's fragmented MP4 made a regular one, as the app does, and nothing else. // -check FILE reads FILE back with FFmpeg and says what it holds, and nothing else: for a // recording cut short by a crash. // -snapshot DIR the last frame captured (captured.png) and a frame decoded back from the file @@ -78,6 +79,12 @@ main :: () { else { print("-capture takes auto, dxgi or wgc\n"); exit(2); } i += 2; continue; } + if a == "-remux" && more >= 2 { + ok, error, packets := record_remux(args[i + 1], args[i + 2]); + if !ok { print("Cannot remux %: %\n", args[i + 1], error); exit(1); } + print("% packets into %\n", packets, args[i + 2]); + exit(0); + } if a == "-check" && more >= 1 { image, info, ok, error := video_poster(args[i + 1]); if !ok { print("%: cannot be read: %\n", args[i + 1], error); exit(1); } diff --git a/video-capture-plan.html b/video-capture-plan.html index 748c75e..6b50c90 100644 --- a/video-capture-plan.html +++ b/video-capture-plan.html @@ -434,6 +434,12 @@ Captured_Frame :: struct {
  • Leftover recordings offered at startup, in a popup in the main window.
  • The Win32 GDI bindings moved to src/base/platform/win32_gdi_bindings.jai, shared by the overlay and the toolbar.
    • +
  • Found by hand: a killed recording's timestamps were wrong (mcap 0.1.7). There were two causes: + + Checked: your recording's MP4, cut mid-fragment, rewrites to a regular MP4 of the 76 whole frames (5.2 s) with positive frame durations. A 12 s mostly still recording has fragments at 0, 2.0, 4.1, 6.1, 8.1, 10.2 and 12.1 s. The app killed about 10 s into a recording leaves 8.1 s. A test cuts a recording at its last fragment, as a crash would, and checks it goes in as a regular MP4 of the time before the cut.
  • Kept out of the recording: a whole-screen recording with the app (-record 6), where the toolbar sits inside the recorded screen. The frame stored in its .mcap has no toolbar. With -record-ui-visible and a region, a screenshot shows both drawn as designed: the frame just outside the region, and the toolbar below it at its right.
  • Pause: 16 s with a 10 s pause from 2 s in gives a 6.2 s video at a steady 30 fps, with Desktop Duplication and with Windows.Graphics.Capture alike. The pause is cut out, with no gap.
  • Losing the screen (-lose, which loses Desktop Duplication every 3 s through the real recovery path): 3 losses in 10 s, 304 frames, 30 fps throughout. A real UAC prompt or resolution change is still to try by hand.