Compare commits

..

No commits in common. "4acde4d7a681b1fb92f80ae892be30becacdfe03" and "59c5b8dd6265c88df090ab538b97349b7e50f968" have entirely different histories.

17 changed files with 86 additions and 1168 deletions

View File

@ -187,14 +187,10 @@ register the key, the Event Log says so; choose another, such as `Ctrl-Shift-S
go of it starts nothing; drag again. go of it starts nothing; drag again.
- Escape or a right click cancels. - Escape or a right click cancels.
3. **Recording starts** at once, at up to 30 frames a second, cursor included. A frame is written 3. **Recording starts** at once, at up to 30 frames a second, cursor included. A frame is written
only when the screen changes, so a still screen costs almost nothing. **Sound:** what the only when the screen changes, so a still screen costs almost nothing.
computer plays is recorded with it, and the microphone once you turn it on. The toolbar's
**System sound** and **Mic** buttons turn each on and off, mid-recording too, and their state is
kept for the next recording.
- **A red border** is drawn just outside the region. - **A red border** is drawn just outside the region.
- **A toolbar** beside it shows the time, the size and any frames dropped, with **System sound** - **A toolbar** beside it shows the time, the size and any frames dropped, with **Pause** (the time
and **Mic** (each on or off), **Pause** (the time paused is left out of the video), **Stop**, and **Discard**, which asks paused is left out of the video), **Stop**, and **Discard**, which asks first.
first.
- **Neither is in the recording.** Neither takes the focus from the window being recorded. - **Neither is in the recording.** Neither takes the focus from the window being recorded.
4. **Ctrl+PrintScreen again**, or Stop, stops it. The video goes into **Screen Captures** beside the stills, 4. **Ctrl+PrintScreen again**, or Stop, stops it. The video goes into **Screen Captures** beside the stills,
as an H.264 MP4 in an `.mcap` with the window it was recorded from, and opens in the Capture window. as an H.264 MP4 in an `.mcap` with the window it was recorded from, and opens in the Capture window.
@ -208,15 +204,9 @@ register the key, the Event Log says so; choose another, such as `Ctrl-Shift-S
- A recording that stops by itself keeps what it has, and so does quitting while recording. - A recording that stops by itself keeps what it has, and so does quitting while recording.
- When the screen is lost for a moment (a UAC prompt, the lock screen, a resolution change), the - When the screen is lost for a moment (a UAC prompt, the lock screen, a resolution change), the
recording carries on once it is back. recording carries on once it is back.
- A recording a crash cut short is offered at the next start: into the library, or deleted. It - A recording a crash cut short is offered at the next start: into the library, or deleted.
keeps all but its last two seconds or so. - **Settings:** `[video]` in the settings file: `fps` (30), `cursor` (true), and `capture`: `auto`
- **Settings:** `[video]` in the settings file: (Desktop Duplication, else Windows.Graphics.Capture), `dxgi` or `wgc`.
- `fps` (30) and `cursor` (true).
- `capture`: `auto` (Desktop Duplication, else Windows.Graphics.Capture), `dxgi` or `wgc`.
- `system_audio` (true): what the computer plays, from the default output device. The toolbar's
System sound button sets it.
- `microphone` (false): the default input device. The toolbar's Mic button sets it.
- Sound is AAC at 160 kb/s. A device unplugged mid-recording is picked up again from the new default.
- **PrintScreen while recording** takes the whole screen under the cursor at once, without freezing - **PrintScreen while recording** takes the whole screen under the cursor at once, without freezing
the screens, which would show in the recording. the screens, which would show in the recording.
- **The key** is `record_region` in `[global_hotkeys]`. - **The key** is `record_region` in `[global_hotkeys]`.

View File

@ -29,7 +29,7 @@ APP_NAME :: "mcap";
APP_FOLDER_NAME :: "mcap"; APP_FOLDER_NAME :: "mcap";
GIT_BRANCH :: "check"; GIT_BRANCH :: "check";
GIT_HASH :: "-"; GIT_HASH :: "-";
VERSION :: "0.1.10"; VERSION :: "0.1.6";
BUILD_DATE :: "-"; BUILD_DATE :: "-";
DEBUG :: true; DEBUG :: true;
COMPILE_TESTS :: true; COMPILE_TESTS :: true;

View File

@ -8,7 +8,7 @@ APP_NAME :: "mcap";
APP_FOLDER_NAME :: "mcap"; APP_FOLDER_NAME :: "mcap";
APP_BUNDLE_IDENTIFIER :: "com.example.mcap"; APP_BUNDLE_IDENTIFIER :: "com.example.mcap";
VERSION :: "0.1.10"; VERSION :: "0.1.6";
// Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated // Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated
// builds never write over the ones you run by hand. // builds never write over the ones you run by hand.
build_directory := "bin"; build_directory := "bin";

View File

@ -120,10 +120,8 @@ Format: "PNG"
## Screen recordings ## Screen recordings
A recording made with the mcap app's Ctrl+PrintScreen is a video: its original is the H.264 MP4, and A recording made with the mcap app's Ctrl+PrintScreen is a video: its original is the H.264 MP4 as
its rendered image (kind 2) a frame from it as a PNG, as for an imported video. It is recorded as a recorded, and its rendered image (kind 2) a frame from it as a PNG, as for an imported video. Its
fragmented MP4, so a crash still leaves one that plays, and stored as a regular one, its frames copied
as they are: one header first, with its length and the index of every frame. Its
metadata.json is a still's. Its capture.json has a still's keys, with Format "MP4", and these: metadata.json is a still's. Its capture.json has a still's keys, with Format "MP4", and these:
``` ```
@ -136,9 +134,6 @@ Capture: "DXGI DuplicateOutput1 on NVIDIA GeForce RTX 5080 Laptop GPU"
or "Windows.Graphics.Capture on ..., border off" or "Windows.Graphics.Capture on ..., border off"
DroppedFrames: 0 frames the encoder fell too far behind for DroppedFrames: 0 frames the encoder fell too far behind for
Paused: 10.2 seconds paused, left out of the video; only when it was paused Paused: 10.2 seconds paused, left out of the video; only when it was paused
Audio: true whether it has sound (AAC)
AudioSources: "SAMSUNG (NVIDIA High Definition Audio); Microphone Array (Realtek(R) Audio)"
the devices heard; only when it has sound
``` ```
A recording a crash cut short, put in the library at the next start, has no window, Encoder or A recording a crash cut short, put in the library at the next start, has no window, Encoder or

View File

@ -1,263 +0,0 @@
// Sound for a recording, with WASAPI: what the computer plays (the default output device, in loopback)
// and the microphone (the default input device). Each is asked for in shared mode as 48 kHz stereo
// float, which Windows converts any device's own format to (AUTOCONVERTPCM), so nothing here
// resamples. Where a device will not convert, its own format is taken if it is 48 kHz float, and its
// channels are made stereo here.
//
// Each packet comes with the QueryPerformanceCounter time its first sample was heard, which places it
// on the recording's timeline (module.jai's audio thread mixes them). Loopback sends nothing while
// nothing plays: those gaps are silence. A device unplugged mid-recording (a headset) is opened
// again, from the new default, once a second.
AUDIO_RATE :: 48000;
AUDIO_CHANNELS :: 2;
Audio_Source_Kind :: enum u8 { SYSTEM; MICROPHONE; }
Audio_Source :: struct {
kind: Audio_Source_Kind;
name: string; // "Speakers (Realtek(R) Audio)"; heap
client: *Audio_Client;
capture: *Audio_Capture_Client;
channels: s32; // as the device gives them: 2, unless it would not convert
open: bool;
lost_at: s64; // when it was lost (ticks), to open it again a second on; 0 while open
error: string; // why it could not be opened; heap
}
Audio_Packet :: struct {
samples: [] float; // interleaved stereo, AUDIO_CHANNELS a frame; empty for silence
frames: s64;
ticks: s64; // when its first frame was heard, in QueryPerformanceCounter ticks
silent: bool;
}
// The default device of its kind, started. False with why: none, or it will not open.
audio_open :: (s: *Audio_Source, kind: Audio_Source_Kind) -> ok: bool, error: string {
s.kind = kind;
enumerator: *Device_Enumerator;
hr := audio_CoCreateInstance(*CLSID_MM_DEVICE_ENUMERATOR, null, CLSCTX_ALL, *IID_MM_DEVICE_ENUMERATOR, xx *enumerator);
if FAILED(hr) return false, hresult_text("the audio device list", hr);
defer com_release(*enumerator);
device: *Audio_Device;
flow: s32 = ifx kind == .SYSTEM then E_RENDER else E_CAPTURE;
hr = enumerator.vtable.GetDefaultAudioEndpoint(enumerator, flow, E_CONSOLE, *device);
if FAILED(hr) return false, ifx kind == .SYSTEM then "no sound output device" else "no microphone";
defer com_release(*device);
free(s.name);
s.name = copy_string(audio_device_name(device));
hr = device.vtable.Activate(device, *IID_AUDIO_CLIENT, CLSCTX_ALL, null, xx *s.client);
if FAILED(hr) return false, hresult_text(tprint("% (%)", s.name, "IAudioClient"), hr);
flags: u32 = AUTOCONVERTPCM | SRC_DEFAULT_QUALITY;
if kind == .SYSTEM flags |= STREAMFLAGS_LOOPBACK;
wanted := WAVEFORMATEX.{wFormatTag = WAVE_FORMAT_IEEE_FLOAT, nChannels = AUDIO_CHANNELS, nSamplesPerSec = AUDIO_RATE,
nAvgBytesPerSec = AUDIO_RATE * AUDIO_CHANNELS * 4, nBlockAlign = AUDIO_CHANNELS * 4, wBitsPerSample = 32};
BUFFER :: 2_000_000; // 200 ms, in 100 ns: room for the audio thread being late
hr = s.client.vtable.Initialize(s.client, SHAREMODE_SHARED, flags, BUFFER, 0, *wanted, null);
s.channels = AUDIO_CHANNELS;
if FAILED(hr) {
// No conversion here: the device's own format, if it is 48 kHz float, its channels made stereo after.
mix: *WAVEFORMATEX;
if FAILED(s.client.vtable.GetMixFormat(s.client, *mix)) return false, hresult_text(tprint("% will not give 48 kHz stereo", s.name), hr);
defer audio_CoTaskMemFree(mix);
is_float := mix.wFormatTag == WAVE_FORMAT_IEEE_FLOAT || (mix.wFormatTag == WAVE_FORMAT_EXTENSIBLE && mix.wBitsPerSample == 32);
if !is_float || mix.nSamplesPerSec != AUDIO_RATE
return false, tprint("% gives % Hz %, not 48 kHz float, and will not convert", s.name, mix.nSamplesPerSec, ifx is_float then "float" else "PCM");
com_release(*s.client);
hr = device.vtable.Activate(device, *IID_AUDIO_CLIENT, CLSCTX_ALL, null, xx *s.client);
if FAILED(hr) return false, hresult_text(s.name, hr);
hr = s.client.vtable.Initialize(s.client, SHAREMODE_SHARED, flags & ~(AUTOCONVERTPCM | SRC_DEFAULT_QUALITY), BUFFER, 0, mix, null);
if FAILED(hr) return false, hresult_text(tprint("% (Initialize)", s.name), hr);
s.channels = mix.nChannels;
}
hr = s.client.vtable.GetService(s.client, *IID_AUDIO_CAPTURE_CLIENT, xx *s.capture);
if FAILED(hr) return false, hresult_text(tprint("% (IAudioCaptureClient)", s.name), hr);
hr = s.client.vtable.Start(s.client);
if FAILED(hr) return false, hresult_text(tprint("% (Start)", s.name), hr);
s.open = true;
s.lost_at = 0;
return true, "";
}
// The packets waiting, each as stereo float (temp), oldest first. lost: the device went away
// (unplugged, or the default changed): audio_close, and open it again later.
audio_take :: (s: *Audio_Source, frequency: s64) -> [] Audio_Packet, lost: bool {
packets: [..] Audio_Packet;
packets.allocator = temp;
while true {
next: u32;
hr := s.capture.vtable.GetNextPacketSize(s.capture, *next);
if hresult_is(hr, AUDCLNT_E_DEVICE_INVALIDATED) return packets, true;
if FAILED(hr) || !next break;
data: *u8;
frames, flags: u32;
device_position, qpc_position: u64; // qpc_position: QueryPerformanceCounter time, in 100 ns
hr = s.capture.vtable.GetBuffer(s.capture, *data, *frames, *flags, *device_position, *qpc_position);
if hresult_is(hr, AUDCLNT_E_DEVICE_INVALIDATED) return packets, true;
if FAILED(hr) || hresult_is(hr, AUDCLNT_S_BUFFER_EMPTY) break;
packet: Audio_Packet;
packet.frames = frames;
packet.ticks = cast(s64) (cast(float64) qpc_position * frequency / 10_000_000.0);
packet.silent = (flags & BUFFERFLAGS_SILENT) != 0;
if !packet.silent && frames {
packet.samples = NewArray(cast(s64) frames * AUDIO_CHANNELS, float, initialized = false,, temp);
input := cast(*float) data;
if s.channels == AUDIO_CHANNELS {
memcpy(packet.samples.data, input, cast(s64) frames * AUDIO_CHANNELS * size_of(float));
} else {
// Mono to both sides; more channels than two: the first two (front left and right).
channels := cast(s64) s.channels;
for f: 0..cast(s64) frames - 1 {
left := input[f * channels];
right := ifx channels > 1 then input[f * channels + 1] else left;
packet.samples[f * 2] = left;
packet.samples[f * 2 + 1] = right;
}
}
}
s.capture.vtable.ReleaseBuffer(s.capture, frames);
array_add(*packets, packet);
}
return packets, false;
}
audio_close :: (s: *Audio_Source) {
if s.client && s.open s.client.vtable.Stop(s.client);
com_release(*s.capture);
com_release(*s.client);
s.open = false;
}
audio_free :: (s: *Audio_Source) {
audio_close(s);
free(s.name);
free(s.error);
s.name, s.error = "", "";
}
// COM on this thread, in the multithreaded apartment, as WASAPI's objects are used from it only.
audio_thread_start :: () {
audio_CoInitializeEx(null, COINIT_MULTITHREADED);
}
audio_thread_end :: () {
audio_CoUninitialize();
}
#scope_file
audio_device_name :: (device: *Audio_Device) -> string {
store: *Property_Store;
if FAILED(device.vtable.OpenPropertyStore(device, STGM_READ, *store)) return "";
defer com_release(*store);
value: PROPVARIANT;
if FAILED(store.vtable.GetValue(store, *PKEY_DEVICE_FRIENDLY_NAME, *value)) return "";
defer audio_PropVariantClear(*value);
VT_LPWSTR :: 31;
if value.vt != VT_LPWSTR || !value.pointer return "";
name, ok := wide_to_utf8(cast(*u16) value.pointer,, temp);
return ifx ok then name else "";
}
// ---------------------------------------------------------------- bindings
ole32_dll :: #library,system "ole32";
audio_CoInitializeEx :: (reserved: *void, flags: u32) -> HRESULT #foreign ole32_dll "CoInitializeEx";
audio_CoUninitialize :: () #foreign ole32_dll "CoUninitialize";
audio_CoCreateInstance :: (clsid: *GUID, outer: *void, class_context: u32, iid: *GUID, out: **void) -> HRESULT #foreign ole32_dll "CoCreateInstance";
audio_CoTaskMemFree :: (memory: *void) #foreign ole32_dll "CoTaskMemFree";
audio_PropVariantClear :: (value: *PROPVARIANT) -> HRESULT #foreign ole32_dll "PropVariantClear";
COINIT_MULTITHREADED : u32 : 0;
CLSCTX_ALL : u32 : 0x17;
E_RENDER : s32 : 0;
E_CAPTURE : s32 : 1;
E_CONSOLE : s32 : 0;
STGM_READ : u32 : 0;
SHAREMODE_SHARED : s32 : 0;
STREAMFLAGS_LOOPBACK : u32 : 0x00020000;
AUTOCONVERTPCM : u32 : 0x80000000;
SRC_DEFAULT_QUALITY : u32 : 0x08000000;
BUFFERFLAGS_SILENT : u32 : 0x2;
WAVE_FORMAT_IEEE_FLOAT : u16 : 3;
WAVE_FORMAT_EXTENSIBLE : u16 : 0xFFFE;
AUDCLNT_S_BUFFER_EMPTY :: 0x08890001;
AUDCLNT_E_DEVICE_INVALIDATED :: 0x88890004;
CLSID_MM_DEVICE_ENUMERATOR := #run guid("BCDE0395-E52F-467C-8E3D-C4579291692E");
IID_MM_DEVICE_ENUMERATOR := #run guid("A95664D2-9614-4F35-A746-DE8DB63617E6");
IID_AUDIO_CLIENT := #run guid("1CB9AD4C-DBFA-4C32-B178-C2F568A703B2");
IID_AUDIO_CAPTURE_CLIENT := #run guid("C8ADBD64-E71E-48A0-A4DE-185C395CD317");
PROPERTYKEY :: struct { fmtid: GUID; pid: u32; }
PKEY_DEVICE_FRIENDLY_NAME := PROPERTYKEY.{#run guid("A45C254E-DF1C-4EFD-8020-67D146A850E0"), 14};
// A PROPVARIANT, as far as a string in it: its type, then (after 6 reserved bytes) the pointer.
PROPVARIANT :: struct {
vt: u16;
reserved: [3] u16;
pointer: *void;
padding: *void;
}
WAVEFORMATEX :: struct {
wFormatTag: u16;
nChannels: u16;
nSamplesPerSec: u32;
nAvgBytesPerSec: u32;
nBlockAlign: u16;
wBitsPerSample: u16;
cbSize: u16;
}
Unknown_Methods :: struct {
QueryInterface: (this: *void, iid: *GUID, out: **void) -> HRESULT #c_call;
AddRef: (this: *void) -> u32 #c_call;
Release: (this: *void) -> u32 #c_call;
}
Device_Enumerator :: struct { vtable: *struct {
using base: Unknown_Methods;
EnumAudioEndpoints: *void;
GetDefaultAudioEndpoint: (this: *Device_Enumerator, flow: s32, role: s32, out: **Audio_Device) -> HRESULT #c_call;
}; }
Audio_Device :: struct { vtable: *struct {
using base: Unknown_Methods;
Activate: (this: *Audio_Device, iid: *GUID, class_context: u32, parameters: *void, out: **void) -> HRESULT #c_call;
OpenPropertyStore: (this: *Audio_Device, access: u32, out: **Property_Store) -> HRESULT #c_call;
}; }
Property_Store :: struct { vtable: *struct {
using base: Unknown_Methods;
GetCount: *void;
GetAt: *void;
GetValue: (this: *Property_Store, key: *PROPERTYKEY, out: *PROPVARIANT) -> HRESULT #c_call;
}; }
Audio_Client :: struct { vtable: *struct {
using base: Unknown_Methods;
Initialize: (this: *Audio_Client, share_mode: s32, flags: u32, buffer_duration: s64, periodicity: s64, format: *WAVEFORMATEX, session: *GUID) -> HRESULT #c_call;
GetBufferSize: *void;
GetStreamLatency: *void;
GetCurrentPadding: *void;
IsFormatSupported: *void;
GetMixFormat: (this: *Audio_Client, out: **WAVEFORMATEX) -> HRESULT #c_call;
GetDevicePeriod: *void;
Start: (this: *Audio_Client) -> HRESULT #c_call;
Stop: (this: *Audio_Client) -> HRESULT #c_call;
Reset: *void;
SetEventHandle: *void;
GetService: (this: *Audio_Client, iid: *GUID, out: **void) -> HRESULT #c_call;
}; }
Audio_Capture_Client :: struct { vtable: *struct {
using base: Unknown_Methods;
GetBuffer: (this: *Audio_Capture_Client, data: **u8, frames: *u32, flags: *u32, device_position: *u64, qpc_position: *u64) -> HRESULT #c_call;
ReleaseBuffer: (this: *Audio_Capture_Client, frames: u32) -> HRESULT #c_call;
GetNextPacketSize: (this: *Audio_Capture_Client, out: *u32) -> HRESULT #c_call;
}; }

View File

@ -78,26 +78,6 @@ gpu_thread_dpi_aware :: () {
user32_dll :: #library,system "user32"; user32_dll :: #library,system "user32";
SetThreadDpiAwarenessContext :: (value: HANDLE) -> HANDLE #foreign user32_dll; SetThreadDpiAwarenessContext :: (value: HANDLE) -> HANDLE #foreign user32_dll;
// A GUID from its text, "A95664D2-9614-4F35-A746-DE8DB63617E6": for #run.
guid :: (text: string) -> GUID {
hex :: (s: string) -> u64 {
value: u64;
for 0..s.count - 1 {
c := s[it];
digit: u64 = ifx c >= #char "a" then cast(u64) (c - #char "a" + 10) else ifx c >= #char "A" then cast(u64) (c - #char "A" + 10) else cast(u64) (c - #char "0");
value = value * 16 + digit;
}
return value;
}
g: GUID;
g.Data1 = cast(u32) hex(slice(text, 0, 8));
g.Data2 = cast(u16) hex(slice(text, 9, 4));
g.Data3 = cast(u16) hex(slice(text, 14, 4));
for 0..1 g.Data4[it] = cast(u8) hex(slice(text, 19 + it * 2, 2));
for 0..5 g.Data4[it + 2] = cast(u8) hex(slice(text, 24 + it * 2, 2));
return g;
}
com_release :: (p: *$T) { com_release :: (p: *$T) {
if p.* { IUnknown_Release(cast(*IUnknown) p.*); p.* = null; } if p.* { IUnknown_Release(cast(*IUnknown) p.*); p.* = null; }
} }

View File

@ -4,17 +4,10 @@
// BGRA as it is and converts it on the GPU; the others are given NV12 or YUV 4:2:0, converted with // BGRA as it is and converts it on the GPU; the others are given NV12 or YUV 4:2:0, converted with
// swscale (BT.709, limited range, as tagged). // swscale (BT.709, limited range, as tagged).
// //
// With sound, an AAC stream (FFmpeg's own encoder, 160 kb/s, 48 kHz stereo) beside it, written from the // The file is written fragmented (a fragment each keyframe, every 2 s), so one cut short by a crash
// audio thread: the two streams' packets meet in the file under a lock (mux). // still plays up to its last fragment.
//
// The file is written fragmented, a fragment at each keyframe, so one cut short by a crash still plays
// up to its last fragment. Frames are written only when the screen changes, so a count of frames can
// span any time: a keyframe is made every RECORD_KEYFRAME_SECONDS of time instead, which bounds what a
// crash loses. remux.jai makes the file a regular MP4 once it is done.
RECORD_ENCODERS :: string.["h264_nvenc", "h264_amf", "h264_qsv", "h264_mf", "libx264"]; RECORD_ENCODERS :: string.["h264_nvenc", "h264_amf", "h264_qsv", "h264_mf", "libx264"];
RECORD_KEYFRAME_SECONDS :: 2;
RECORD_AUDIO_BIT_RATE :: 160_000;
Encoder :: struct { Encoder :: struct {
format: *AVFormatContext; format: *AVFormatContext;
@ -27,23 +20,13 @@ Encoder :: struct {
input: AVPixelFormat; // what it is given: BGR0 as captured, or converted to NV12 or YUV420P input: AVPixelFormat; // what it is given: BGR0 as captured, or converted to NV12 or YUV420P
width, height: s32; width, height: s32;
last_pts: s64 = -1; last_pts: s64 = -1;
last_key_pts: s64 = -1; // the last keyframe asked for
header_written: bool; header_written: bool;
bytes: s64; // of packets written bytes: s64; // of packets written
// Sound, when asked for: AAC, written from the audio thread.
audio: *AVCodecContext;
audio_stream: *AVStream;
audio_frame: *AVFrame;
audio_packet: *AVPacket;
audio_frame_size: s32; // samples a frame: AAC's 1024
mux: Mutex; // the video and audio threads' packets into the file
mux_ready: bool;
} }
// Opens the file and the first encoder that will take width x height (wanted alone, when given). // Opens the file and the first encoder that will take width x height (wanted alone, when given).
// Failures to open an encoder are listed in tried (temp), for the log. // Failures to open an encoder are listed in tried (temp), for the log.
encoder_open :: (e: *Encoder, path: string, width: s32, height: s32, fps: s32, wanted: string, audio := false) -> ok: bool, error: string, tried: string { encoder_open :: (e: *Encoder, path: string, width: s32, height: s32, fps: s32, wanted: string) -> ok: bool, error: string, tried: string {
av_log_set_level(AV_LOG_ERROR); av_log_set_level(AV_LOG_ERROR);
e.width, e.height = width, height; e.width, e.height = width, height;
result := avformat_alloc_output_context2(*e.format, null, "mp4", temp_c_string(path)); result := avformat_alloc_output_context2(*e.format, null, "mp4", temp_c_string(path));
@ -68,12 +51,6 @@ encoder_open :: (e: *Encoder, path: string, width: s32, height: s32, fps: s32, w
if !e.stream { encoder_free(e); return false, "FFmpeg cannot add a stream", tried_text; } if !e.stream { encoder_free(e); return false, "FFmpeg cannot add a stream", tried_text; }
avcodec_parameters_from_context(e.stream.codecpar, e.codec); avcodec_parameters_from_context(e.stream.codecpar, e.codec);
e.stream.time_base = e.codec.time_base; e.stream.time_base = e.codec.time_base;
if audio {
audio_ok, audio_error := encoder_open_audio(e);
if !audio_ok { encoder_free(e); return false, audio_error, tried_text; }
}
init(*e.mux, "record mux");
e.mux_ready = true;
result = avio_open(*e.format.pb, temp_c_string(path), AVIO_FLAG_WRITE); result = avio_open(*e.format.pb, temp_c_string(path), AVIO_FLAG_WRITE);
if result < 0 { encoder_free(e); return false, av_error(tprint("FFmpeg cannot write %", path), result), tried_text; } if result < 0 { encoder_free(e); return false, av_error(tprint("FFmpeg cannot write %", path), result), tried_text; }
@ -118,10 +95,6 @@ encoder_write :: (e: *Encoder, pixels: *u8, pts: s64) -> ok: bool, error: string
sws_scale(e.sws, *source, *stride, 0, e.height, frame.data.data, frame.linesize.data); sws_scale(e.sws, *source, *stride, 0, e.height, frame.data.data, frame.linesize.data);
} }
frame.pts = pts_out; frame.pts = pts_out;
if e.last_key_pts < 0 || pts_out - e.last_key_pts >= RECORD_KEYFRAME_SECONDS * RECORD_TIME_BASE {
frame.pict_type = .I; // an IDR, with forced-idr: a new fragment
e.last_key_pts = pts_out;
}
result := avcodec_send_frame(e.codec, frame); result := avcodec_send_frame(e.codec, frame);
av_frame_unref(frame); av_frame_unref(frame);
@ -130,36 +103,12 @@ encoder_write :: (e: *Encoder, pixels: *u8, pts: s64) -> ok: bool, error: string
return drained, error; return drained, error;
} }
// One AAC frame of sound: e.audio_frame_size frames of interleaved stereo float, at pts in samples. // Flushes the encoder, finishes the file and closes it.
// From the audio thread.
encoder_write_audio :: (e: *Encoder, samples: *float, pts: s64) -> ok: bool, error: string {
frame := e.audio_frame;
result := av_frame_make_writable(frame);
if result < 0 return false, av_error("FFmpeg cannot make a sound frame", result);
left := cast(*float) frame.data[0];
right := cast(*float) frame.data[1];
for 0..e.audio_frame_size - 1 {
left[it] = samples[it * 2];
right[it] = samples[it * 2 + 1];
}
frame.pts = pts;
result = avcodec_send_frame(e.audio, frame);
if result < 0 return false, av_error("the AAC encoder", result);
drained, error := encoder_drain(e, e.audio, e.audio_stream, e.audio_packet);
return drained, error;
}
// Flushes the encoders, sound first, finishes the file and closes it.
encoder_close :: (e: *Encoder) -> ok: bool, error: string { encoder_close :: (e: *Encoder) -> ok: bool, error: string {
ok, error := true, ""; ok, error := true, "";
if e.audio && e.header_written {
avcodec_send_frame(e.audio, null);
ok, error = encoder_drain(e, e.audio, e.audio_stream, e.audio_packet);
}
if e.codec && e.header_written { if e.codec && e.header_written {
avcodec_send_frame(e.codec, null); avcodec_send_frame(e.codec, null);
video_ok, video_error := encoder_drain(e); ok, error = encoder_drain(e);
if ok ok, error = video_ok, video_error;
result := av_write_trailer(e.format); result := av_write_trailer(e.format);
if ok && result < 0 ok, error = false, av_error("FFmpeg cannot finish the MP4", result); if ok && result < 0 ok, error = false, av_error("FFmpeg cannot finish the MP4", result);
} }
@ -198,11 +147,11 @@ encoder_try :: (e: *Encoder, name: string, width: s32, height: s32, fps: s32) ->
options: *AVDictionary; options: *AVDictionary;
set :: (key: string, value: string) #expand { av_dict_set(*`options, temp_c_string(key), temp_c_string(value), 0); } set :: (key: string, value: string) #expand { av_dict_set(*`options, temp_c_string(key), temp_c_string(value), 0); }
if name == { if name == {
case "h264_nvenc"; set("preset", "p4"); set("rc", "vbr"); set("cq", "23"); set("forced-idr", "1"); c.bit_rate = 0; case "h264_nvenc"; set("preset", "p4"); set("rc", "vbr"); set("cq", "23"); c.bit_rate = 0;
case "h264_amf"; set("rc", "cqp"); set("qp_i", "22"); set("qp_p", "24"); set("quality", "balanced"); case "h264_amf"; set("rc", "cqp"); set("qp_i", "22"); set("qp_p", "24"); set("quality", "balanced");
case "h264_qsv"; set("global_quality", "23"); set("preset", "medium"); case "h264_qsv"; set("global_quality", "23"); set("preset", "medium");
case "h264_mf"; set("rate_control", "quality"); set("quality", "70"); set("scenario", "display_remoting"); case "h264_mf"; set("rate_control", "quality"); set("quality", "70"); set("scenario", "display_remoting");
case "libx264"; set("preset", "veryfast"); set("crf", "23"); set("forced-idr", "1"); case "libx264"; set("preset", "veryfast"); set("crf", "23");
} }
result := avcodec_open2(c, codec, *options); result := avcodec_open2(c, codec, *options);
av_dict_free(*options); av_dict_free(*options);
@ -229,64 +178,22 @@ encoder_input_format :: (codec: *AVCodec) -> AVPixelFormat {
return .NONE; return .NONE;
} }
// The packets an encoder (the video's, unless another is given) has ready, into the file. encoder_drain :: (e: *Encoder) -> ok: bool, error: string {
encoder_drain :: (e: *Encoder, codec: *AVCodecContext = null, stream: *AVStream = null, packet: *AVPacket = null) -> ok: bool, error: string {
AVERROR_EAGAIN :: -11; AVERROR_EAGAIN :: -11;
if !codec codec, stream, packet = e.codec, e.stream, e.packet;
while true { while true {
result := avcodec_receive_packet(codec, packet); result := avcodec_receive_packet(e.codec, e.packet);
if result == AVERROR_EAGAIN || result == cast,no_check(s32) AVERROR_EOF return true, ""; if result == AVERROR_EAGAIN || result == cast,no_check(s32) AVERROR_EOF return true, "";
if result < 0 return false, av_error(tprint("% failed", ifx codec == e.audio then "the AAC encoder" else e.name), result); if result < 0 return false, av_error(tprint("% failed", e.name), result);
av_packet_rescale_ts(packet, codec.time_base, stream.time_base); av_packet_rescale_ts(e.packet, e.codec.time_base, e.stream.time_base);
packet.stream_index = stream.index; e.packet.stream_index = e.stream.index;
lock(*e.mux); e.bytes += e.packet.size;
e.bytes += packet.size; result = av_interleaved_write_frame(e.format, e.packet); // takes the packet's data
result = av_interleaved_write_frame(e.format, packet); // takes the packet's data
unlock(*e.mux);
if result < 0 return false, av_error("FFmpeg cannot write to the MP4", result); if result < 0 return false, av_error("FFmpeg cannot write to the MP4", result);
} }
return true, ""; return true, "";
} }
encoder_open_audio :: (e: *Encoder) -> ok: bool, error: string {
codec := avcodec_find_encoder_by_name("aac");
if !codec return false, "this FFmpeg has no AAC encoder";
c := avcodec_alloc_context3(codec);
if !c return false, "out of memory";
c.sample_fmt = .FLTP;
c.sample_rate = AUDIO_RATE;
av_channel_layout_default(*c.ch_layout, AUDIO_CHANNELS);
c.bit_rate = RECORD_AUDIO_BIT_RATE;
c.time_base = .{1, AUDIO_RATE};
if e.format.oformat.flags & AVFMT_GLOBALHEADER c.flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
result := avcodec_open2(c, codec, null);
if result < 0 { avcodec_free_context(*c); return false, av_error("the AAC encoder cannot open", result); }
e.audio = c;
e.audio_frame_size = ifx c.frame_size > 0 then c.frame_size else 1024;
e.audio_stream = avformat_new_stream(e.format, null);
if !e.audio_stream return false, "FFmpeg cannot add a sound stream";
avcodec_parameters_from_context(e.audio_stream.codecpar, c);
e.audio_stream.time_base = c.time_base;
e.audio_packet = av_packet_alloc();
e.audio_frame = av_frame_alloc();
if !e.audio_packet || !e.audio_frame return false, "FFmpeg is out of memory";
e.audio_frame.format = cast(s32) AVSampleFormat.FLTP;
e.audio_frame.nb_samples = e.audio_frame_size;
e.audio_frame.sample_rate = AUDIO_RATE;
av_channel_layout_default(*e.audio_frame.ch_layout, AUDIO_CHANNELS);
result = av_frame_get_buffer(e.audio_frame, 0);
if result < 0 return false, av_error("FFmpeg cannot make a sound frame", result);
return true, "";
}
encoder_free :: (e: *Encoder) { encoder_free :: (e: *Encoder) {
if e.audio_frame av_frame_free(*e.audio_frame);
if e.audio_packet av_packet_free(*e.audio_packet);
if e.audio avcodec_free_context(*e.audio);
e.audio_stream = null;
if e.mux_ready { destroy(*e.mux); e.mux_ready = false; }
if e.sws sws_freeContext(e.sws); if e.sws sws_freeContext(e.sws);
if e.frame av_frame_free(*e.frame); if e.frame av_frame_free(*e.frame);
if e.packet av_packet_free(*e.packet); if e.packet av_packet_free(*e.packet);
@ -298,8 +205,6 @@ encoder_free :: (e: *Encoder) {
e.sws, e.format, e.stream = null, null, null; e.sws, e.format, e.stream = null, null, null;
} }
#scope_module
// What FFmpeg says an error code is, after what failed (temp). // What FFmpeg says an error code is, after what failed (temp).
av_error :: (what: string, code: s32) -> string { av_error :: (what: string, code: s32) -> string {
buffer: [256] u8; buffer: [256] u8;

View File

@ -13,13 +13,6 @@
// one a second while nothing does, so players can seek. Each carries the time it was taken, less the // one a second while nothing does, so players can seek. Each carries the time it was taken, less the
// time spent paused (record_pause), so a pause leaves no gap in the video. // time spent paused (record_pause), so a pause leaves no gap in the video.
// //
// Sound (system_audio, microphone) has a thread of its own (audio_source.jai's WASAPI): each packet is
// put on the same timeline as the frames, by when it was heard, less the same pauses, into one mix,
// and an AAC frame of it is encoded once every source has had its say up to there. A source that sends
// nothing (the computer playing nothing) is silence. Either source can be turned on and off while
// recording (record_set_audio); with audio_track, the MP4 has a sound track even when no source is on
// at the start, so one can be.
//
// recorder: Recorder; // recorder: Recorder;
// ok, error, fall_back := record_start(*recorder, .{path = "out.mp4", screen = 0}); // ok, error, fall_back := record_start(*recorder, .{path = "out.mp4", screen = 0});
// ... // ...
@ -73,9 +66,6 @@ Record_Options :: struct {
capture: Record_Capture; capture: Record_Capture;
legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe
testing: Record_Testing; testing: Record_Testing;
system_audio: bool; // what the computer plays (the default output device)
microphone: bool; // the default input device; both at the start: record_set_audio turns them on and off
audio_track: bool; // a sound track whatever is on at the start, so a source can be turned on later
} }
Record_Stats :: struct { Record_Stats :: struct {
@ -91,9 +81,6 @@ Record_Stats :: struct {
dropped: s64; // the encoder was too far behind dropped: s64; // the encoder was too far behind
lost: s64; // times the duplication was lost and made again lost: s64; // times the duplication was lost and made again
paused_seconds: float64; // left out of the video paused_seconds: float64; // left out of the video
audio: string; // the sound recorded: "Speakers (Realtek(R) Audio); Microphone (USB Audio)", or ""
audio_frames: s64; // AAC frames encoded
audio_errors: string; // sources that could not be opened, or were lost
copy_ticks, readback_ticks, encode_ticks: s64; // totals, in QueryPerformanceCounter ticks copy_ticks, readback_ticks, encode_ticks: s64; // totals, in QueryPerformanceCounter ticks
bytes: s64; // encoded bytes: s64; // encoded
seconds: float64; // from start to stop seconds: float64; // from start to stop
@ -112,17 +99,7 @@ Recorder :: struct {
start_ticks: s64; start_ticks: s64;
capture_thread, encoder_thread: Thread; capture_thread, encoder_thread: Thread;
capture_started, encoder_started, audio_started: bool; capture_started, encoder_started: bool;
audio_thread: Thread;
audio_opened: Semaphore; // the audio thread has opened its sources
audio_go: Semaphore; // the encoder is open: the audio thread may write
audio_sources: [2] Audio_Source;
audio_open: bool; // any source opened, or audio_track: the MP4 has sound
audio_on: [2] bool; // each source is wanted (by Audio_Source_Kind): record_set_audio; cleared when it cannot be opened
audio_failed: [2] bool; // it could not be opened after being turned on: record_audio_failed says once
zero_ticks: s64; // the recording's time 0, for frames and sound alike
paused_ticks: s64; // time paused so far, for the capture and audio threads; set by record_pause
pause_began: s64;
running: bool; running: bool;
stopping: bool; // set by record_stop; the capture thread ends, then the encoder stopping: bool; // set by record_stop; the capture thread ends, then the encoder
paused: bool; // set by record_pause; the capture thread takes nothing while it is paused: bool; // set by record_pause; the capture thread takes nothing while it is
@ -198,7 +175,6 @@ record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: stri
QueryPerformanceFrequency(*frequency); QueryPerformanceFrequency(*frequency);
r.frequency = frequency.QuadPart; r.frequency = frequency.QuadPart;
r.start_ticks = ticks_now(); r.start_ticks = ticks_now();
r.zero_ticks = r.start_ticks;
// The source is opened on the capture thread, which uses it: WinRT's objects belong to the // The source is opened on the capture thread, which uses it: WinRT's objects belong to the
// apartment they are made in. // apartment they are made in.
@ -212,19 +188,7 @@ record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: stri
return false, error, fall_back; return false, error, fall_back;
} }
// The sound's sources are opened on their own thread (COM, for WASAPI), before the encoder, which ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder);
// has a sound stream only when one opened.
r.audio_on = .[r.options.system_audio, r.options.microphone];
if r.options.system_audio || r.options.microphone || r.options.audio_track {
init(*r.audio_opened);
init(*r.audio_go);
thread_init(*r.audio_thread, record_audio_proc, name = "record audio");
r.audio_started = true;
thread_start(*r.audio_thread, r);
wait_for(*r.audio_opened);
}
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder, audio = r.audio_open);
if tried log("[record] Encoders that did not open: %", tried); if tried log("[record] Encoders that did not open: %", tried);
if !ok { if !ok {
error = copy_string(error,, temp); error = copy_string(error,, temp);
@ -232,7 +196,6 @@ record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: stri
return false, error, false; return false, error, false;
} }
r.stats.encoder = copy_string(r.encoder.name); r.stats.encoder = copy_string(r.encoder.name);
if r.audio_started signal(*r.audio_go);
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder"); thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
r.encoder_started = true; r.encoder_started = true;
@ -248,7 +211,6 @@ record_stop :: (r: *Recorder) -> ok: bool, error: string {
atomic_swap(*r.stopping, true); atomic_swap(*r.stopping, true);
thread_is_done(*r.capture_thread, -1); thread_is_done(*r.capture_thread, -1);
thread_is_done(*r.encoder_thread, -1); thread_is_done(*r.encoder_thread, -1);
if r.audio_started thread_is_done(*r.audio_thread, -1);
r.running = false; r.running = false;
closed, close_error := encoder_close(*r.encoder); closed, close_error := encoder_close(*r.encoder);
@ -262,44 +224,9 @@ record_stop :: (r: *Recorder) -> ok: bool, error: string {
return true, ""; return true, "";
} }
// Pauses or resumes. While paused nothing is taken, and the time is left out of the video, sound and // Pauses or resumes. While paused nothing is taken, and the time is left out of the video.
// all: the time paused is kept here, once, for the capture and audio threads both.
record_pause :: (r: *Recorder, pause: bool) { record_pause :: (r: *Recorder, pause: bool) {
if !r.running || atomic_read(*r.paused) == pause return; if r.running atomic_swap(*r.paused, pause);
now := ticks_now();
if pause {
atomic_write(*r.pause_began, now);
atomic_swap(*r.paused, true);
} else {
atomic_add(*r.paused_ticks, now - atomic_read(*r.pause_began)); // before paused clears, so no thread sees it unpaused with the old total
atomic_swap(*r.paused, false);
}
}
// Turns a source of sound on or off while recording. It is opened only while on, so Windows shows the
// microphone in use only then. False when the recording has no sound track to put it in.
record_set_audio :: (r: *Recorder, kind: Audio_Source_Kind, on: bool) -> bool {
if !r.running || !r.audio_open return false;
atomic_swap(*r.audio_on[kind], on);
return true;
}
record_audio_on :: (r: *Recorder, kind: Audio_Source_Kind) -> bool {
return r.running && atomic_read(*r.audio_on[kind]);
}
// Whether a source could not be opened when turned on (it is off again then), and why; once.
record_audio_failed :: (r: *Recorder, kind: Audio_Source_Kind) -> failed: bool, error: string {
if !r.running || !atomic_read(*r.audio_failed[kind]) return false, "";
atomic_swap(*r.audio_failed[kind], false);
lock(*r.mutex);
defer unlock(*r.mutex);
return true, copy_string(r.stats.audio_errors,, temp);
}
// Ticks on the recording's timeline: time 0 at its start, less the time paused.
record_timeline :: (r: *Recorder, ticks: s64) -> s64 {
return ticks - r.zero_ticks - atomic_read(*r.paused_ticks);
} }
record_paused :: (r: *Recorder) -> bool { record_paused :: (r: *Recorder) -> bool {
@ -338,14 +265,6 @@ record_free :: (r: *Recorder) {
atomic_swap(*r.stopping, true); atomic_swap(*r.stopping, true);
if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; } if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; }
if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; } if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; }
if r.audio_started {
signal(*r.audio_go); // in case it waits for an encoder that did not open
thread_is_done(*r.audio_thread, -1);
thread_deinit(*r.audio_thread);
destroy(*r.audio_opened);
destroy(*r.audio_go);
r.audio_started = false;
}
if r.encoder.format encoder_close(*r.encoder); if r.encoder.format encoder_close(*r.encoder);
for * r.frames array_free(it.pixels); for * r.frames array_free(it.pixels);
array_free(r.last_frame); array_free(r.last_frame);
@ -412,7 +331,7 @@ record_capture_proc :: (thread: *Thread) -> s64 {
have_image, changed := false, false; have_image, changed := false, false;
staged: [RECORD_STAGING] Staged; staged: [RECORD_STAGING] Staged;
emits: s64; emits: s64;
was_paused := false; paused_ticks, pause_start: s64; // time paused so far, left out of the frames' times; when this pause began
last_lost := ticks_now(); // for Record_Testing.LOSE_DUPLICATION last_lost := ticks_now(); // for Record_Testing.LOSE_DUPLICATION
while !atomic_read(*r.stopping) { while !atomic_read(*r.stopping) {
@ -462,15 +381,15 @@ record_capture_proc :: (thread: *Thread) -> s64 {
// screen as it is now is sent at once. // screen as it is now is sent at once.
now = ticks_now(); now = ticks_now();
if atomic_read(*r.paused) { if atomic_read(*r.paused) {
was_paused = true; if !pause_start pause_start = now;
lock(*r.mutex); r.stats.paused_seconds = cast(float64) (atomic_read(*r.paused_ticks) + now - atomic_read(*r.pause_began)) / r.frequency; unlock(*r.mutex); lock(*r.mutex); r.stats.paused_seconds = cast(float64) (paused_ticks + now - pause_start) / r.frequency; unlock(*r.mutex);
continue; continue;
} }
if was_paused { if pause_start {
was_paused = false; paused_ticks += now - pause_start;
pause_start = 0;
changed = true; changed = true;
turn = now; turn = now;
lock(*r.mutex); r.stats.paused_seconds = cast(float64) atomic_read(*r.paused_ticks) / r.frequency; unlock(*r.mutex);
} }
if !have_image continue; if !have_image continue;
repeat := !changed && now - last_emit >= r.frequency; repeat := !changed && now - last_emit >= r.frequency;
@ -481,7 +400,7 @@ record_capture_proc :: (thread: *Thread) -> s64 {
slot := emits % RECORD_STAGING; slot := emits % RECORD_STAGING;
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]); if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
ID3D11DeviceContext_CopyResource(gpu.device_context, gpu.staging[slot], gpu.latest); ID3D11DeviceContext_CopyResource(gpu.device_context, gpu.staging[slot], gpu.latest);
staged[slot] = .{pending = true, ticks = record_timeline(r, now)}; staged[slot] = .{pending = true, ticks = now - paused_ticks};
if !r.using_wgc staged[slot].pointer, staged[slot].pointer_visible = r.dxgi.pointer_position, r.dxgi.pointer_visible; if !r.using_wgc staged[slot].pointer, staged[slot].pointer_visible = r.dxgi.pointer_position, r.dxgi.pointer_visible;
emits += 1; emits += 1;
last_emit = now; last_emit = now;
@ -546,6 +465,7 @@ record_read_back :: (r: *Recorder, gpu: *Capture_Gpu, slot: s64, staged: Staged)
record_encoder_proc :: (thread: *Thread) -> s64 { record_encoder_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data; r := cast(*Recorder) thread.data;
first: s64 = -1;
while true { while true {
wait_for(*r.ready, 100); wait_for(*r.ready, 100);
lock(*r.mutex); lock(*r.mutex);
@ -562,7 +482,8 @@ record_encoder_proc :: (thread: *Thread) -> s64 {
} }
frame := *r.frames[index]; frame := *r.frames[index];
pts := max(frame.ticks, 0) * 90000 / r.frequency; // frame.ticks is on the timeline already if first < 0 first = frame.ticks;
pts := (frame.ticks - first) * 90000 / r.frequency;
t := ticks_now(); t := ticks_now();
ok, error := encoder_write(*r.encoder, frame.pixels.data, pts); ok, error := encoder_write(*r.encoder, frame.pixels.data, pts);
@ -589,171 +510,6 @@ record_encoder_proc :: (thread: *Thread) -> s64 {
return 0; return 0;
} }
// The audio thread: opens the sources asked for, waits for the encoder, then every 10 ms takes their
// packets into the mix and encodes what every source has covered.
record_audio_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
audio_thread_start();
defer audio_thread_end();
kinds: [2] Audio_Source_Kind = .[.SYSTEM, .MICROPHONE];
wanted: [2] bool = .[r.options.system_audio, r.options.microphone];
names, errors: [..] string; // every source heard, and every failure, for the stats
names.allocator, errors.allocator = context.default_allocator, context.default_allocator;
defer { for names free(it); array_free(names); for errors free(it); array_free(errors); }
note :: (list: *[..] string, text: string, field: *string) #expand {
for list.* if it == text return;
array_add(list, copy_string(text));
lock(*`r.mutex);
free(field.*);
field.* = join(..list.*, "; ");
unlock(*`r.mutex);
}
for 0..1 {
if !wanted[it] continue;
s := *r.audio_sources[it];
ok, error := audio_open(s, kinds[it]);
what := ifx kinds[it] == .SYSTEM then "system sound" else "microphone";
if ok { r.audio_open = true; note(*names, s.name, *r.stats.audio); }
else { audio_close(s); note(*errors, tprint("%: %", what, error), *r.stats.audio_errors); log_error("[record] No %: %", what, error); }
}
for r.audio_sources if !it.open atomic_swap(*r.audio_on[it_index], false);
if r.options.audio_track r.audio_open = true;
signal(*r.audio_opened);
wait_for(*r.audio_go);
if !r.audio_open || !r.encoder.audio || atomic_read(*r.stopping) {
for * r.audio_sources audio_free(it);
return 0;
}
mix: Audio_Mix;
mix.frame_size = r.encoder.audio_frame_size;
defer array_free(mix.samples);
while true {
stopping := atomic_read(*r.stopping);
// Each source, turned on or off from the toolbar: opened only while on.
for * source, index: r.audio_sources {
on := atomic_read(*r.audio_on[index]) && !stopping;
what := ifx kinds[index] == .SYSTEM then "System sound" else "Microphone";
if on && !source.open && !wanted[index] {
ok, error := audio_open(source, kinds[index]);
if ok {
log("[record] % on: %.", what, source.name);
note(*names, source.name, *r.stats.audio);
mix.started[index] = false;
} else {
audio_close(source);
log_error("[record] % could not be opened: %", what, error);
note(*errors, tprint("%: %", to_lower_copy(what,, temp), error), *r.stats.audio_errors);
atomic_swap(*r.audio_on[index], false);
atomic_swap(*r.audio_failed[index], true);
}
} else if !on && source.open {
log("[record] % off.", what);
audio_close(source);
source.lost_at = 0;
}
wanted[index] = source.open || (wanted[index] && on);
}
for * source, index: r.audio_sources {
if !wanted[index] continue;
if !source.open { // lost: opened again a second on, from the default device then
if source.lost_at && ticks_now() - source.lost_at >= r.frequency && !stopping {
if audio_open(source, kinds[index]) log("[record] % is back: %.", ifx kinds[index] == .SYSTEM then "System sound" else "The microphone", source.name);
else { audio_close(source); source.lost_at = ticks_now(); }
}
continue;
}
packets, lost := audio_take(source, r.frequency);
for packets if !atomic_read(*r.paused) audio_mix_add(*mix, r, index, it);
if lost {
log_error("[record] % was lost (unplugged, or the default device changed); trying again.", source.name);
audio_close(source);
source.lost_at = ticks_now();
}
}
ok, error := audio_mix_encode(*mix, r, final = stopping);
if !ok { record_fail(r, error); break; }
if stopping break;
reset_temporary_storage();
sleep_milliseconds(10);
}
for * r.audio_sources audio_free(it);
return 0;
}
// The sound so far, from its base: one AAC frame's worth is encoded at a time.
Audio_Mix :: struct {
samples: [..] float; // interleaved stereo, from base on
base: s64; // the frame (in samples, on the timeline) samples[0] is
next: [2] s64; // each source's next frame, so packets that follow on are not moved by jitter
started: [2] bool;
frame_size: s32;
}
AUDIO_SLACK :: AUDIO_RATE / 5; // 200 ms: how late a source's packets may come and still be in the mix
AUDIO_FOLLOW_ON :: AUDIO_RATE / 33; // 30 ms: a packet that close to where its source left off follows on
// A packet into the mix at its place on the timeline, added to what the other source put there.
audio_mix_add :: (mix: *Audio_Mix, r: *Recorder, source: s64, packet: Audio_Packet) {
at := record_timeline(r, packet.ticks) * AUDIO_RATE / r.frequency;
if mix.started[source] && abs(at - mix.next[source]) <= AUDIO_FOLLOW_ON at = mix.next[source];
mix.started[source] = true;
mix.next[source] = at + packet.frames;
end := (at + packet.frames - mix.base) * AUDIO_CHANNELS;
if end > mix.samples.count {
old := mix.samples.count;
array_resize(*mix.samples, end, initialize = false);
for old..end - 1 mix.samples[it] = 0;
}
if packet.silent || !packet.samples.count return;
for frame: max(at, mix.base)..at + packet.frames - 1 {
into := (frame - mix.base) * AUDIO_CHANNELS;
from := (frame - at) * AUDIO_CHANNELS;
mix.samples[into] += packet.samples[from];
mix.samples[into + 1] += packet.samples[from + 1];
}
}
// Encodes the frames every open source has covered: up to where each has sent, or, for one that has
// sent nothing lately (silence), to AUDIO_SLACK ago. final: everything sent.
audio_mix_encode :: (mix: *Audio_Mix, r: *Recorder, final: bool) -> ok: bool, error: string {
// Paused, the timeline stands still where the pause began.
now := ifx atomic_read(*r.paused) then atomic_read(*r.pause_began) else ticks_now();
now_frame := record_timeline(r, now) * AUDIO_RATE / r.frequency;
ready := now_frame - AUDIO_SLACK;
for source, index: r.audio_sources {
if !source.open continue;
covered := max(mix.next[index], now_frame - AUDIO_SLACK);
ready = min(ready, covered);
}
if final { ready = mix.base; for mix.next ready = max(ready, it); }
size := mix.frame_size;
frame := NewArray(size * AUDIO_CHANNELS, float, initialized = false,, temp);
done: s64;
while mix.base + size <= ready {
for i: 0..frame.count - 1 {
at := done * AUDIO_CHANNELS + i;
value := ifx at < mix.samples.count then mix.samples[at] else 0.0;
frame[i] = clamp(value, -1.0, 1.0);
}
ok, error := encoder_write_audio(*r.encoder, frame.data, mix.base);
if !ok return false, error;
mix.base += size;
done += size;
lock(*r.mutex); r.stats.audio_frames += 1; unlock(*r.mutex);
}
// What was encoded leaves the front.
gone := min(done * AUDIO_CHANNELS, mix.samples.count);
if gone {
memcpy(mix.samples.data, mix.samples.data + gone, (mix.samples.count - gone) * size_of(float));
mix.samples.count -= gone;
}
return true, "";
}
// Ends the recording from one of its threads; the first reason is kept. // Ends the recording from one of its threads; the first reason is kept.
record_fail :: (r: *Recorder, error: string) { record_fail :: (r: *Recorder, error: string) {
log_error("[record] %", error); log_error("[record] %", error);
@ -773,14 +529,11 @@ rect_inside :: (inner: RECT, outer: RECT) -> bool {
#load "dxgi_source.jai"; #load "dxgi_source.jai";
#load "wgc_source.jai"; #load "wgc_source.jai";
#load "encoder.jai"; #load "encoder.jai";
#load "remux.jai";
#load "audio_source.jai";
#load "cursor.jai"; #load "cursor.jai";
#import "Basic"; #import "Basic";
#import "String"; #import "String";
#import "Atomics"; #import "Atomics";
#import "Math";
#import "Thread"; #import "Thread";
#import "Windows"; #import "Windows";
#import "Windows_Utf8"; #import "Windows_Utf8";

View File

@ -1,65 +0,0 @@
// A recording made into a regular MP4. While recording, the MP4 is written fragmented (encoder.jai), so
// a crash still leaves one that plays; but a fragmented MP4's header says nothing of its length or
// where its frames are, and one cut short by a crash never gets the index written at the end of a
// clean stop, so players show its time and seek in it badly. record_remux copies its packets, as they
// are, into a regular MP4: one header, with the length and the index of every frame, at the front
// (faststart), so it plays, seeks and streams as any MP4 does. Nothing is encoded again.
// Copies source's video and audio packets into a regular MP4 at destination. packets: how many.
record_remux :: (source: string, destination: string) -> ok: bool, error: string, packets: s64 {
av_log_set_level(AV_LOG_ERROR);
input: *AVFormatContext;
result := avformat_open_input(*input, temp_c_string(source), null, null);
if result < 0 return false, av_error("FFmpeg cannot open the recording", result), 0;
defer avformat_close_input(*input);
result = avformat_find_stream_info(input, null);
if result < 0 return false, av_error("FFmpeg cannot read the recording's streams", result), 0;
output: *AVFormatContext;
result = avformat_alloc_output_context2(*output, null, "mp4", temp_c_string(destination));
if result < 0 || !output return false, av_error("FFmpeg cannot make an MP4", result), 0;
defer {
if output.pb avio_closep(*output.pb);
avformat_free_context(output);
}
// Each input stream's output stream, or -1 for one left out (neither video nor audio).
map := NewArray(input.nb_streams, s32,, temp);
for 0..input.nb_streams - 1 {
in := input.streams[it];
map[it] = -1;
if in.codecpar.codec_type != .VIDEO && in.codecpar.codec_type != .AUDIO continue;
out := avformat_new_stream(output, null);
if !out return false, "FFmpeg cannot add a stream", 0;
avcodec_parameters_copy(out.codecpar, in.codecpar);
out.codecpar.codec_tag = 0; // the MP4 muxer picks its own
out.time_base = in.time_base;
map[it] = out.index;
}
result = avio_open(*output.pb, temp_c_string(destination), AVIO_FLAG_WRITE);
if result < 0 return false, av_error(tprint("FFmpeg cannot write %", destination), result), 0;
options: *AVDictionary;
av_dict_set(*options, "movflags", "+faststart", 0); // the header before the frames: a second pass at the end
result = avformat_write_header(output, *options);
av_dict_free(*options);
if result < 0 return false, av_error("FFmpeg cannot start the MP4", result), 0;
packet := av_packet_alloc();
defer av_packet_free(*packet);
packets: s64;
while av_read_frame(input, packet) >= 0 { // to the end, or to where a crash cut it
index := map[packet.stream_index];
if index < 0 { av_packet_unref(packet); continue; }
av_packet_rescale_ts(packet, input.streams[packet.stream_index].time_base, output.streams[index].time_base);
packet.stream_index = index;
packet.pos = -1;
result = av_interleaved_write_frame(output, packet); // takes the packet's data
if result < 0 return false, av_error("FFmpeg cannot write to the MP4", result), packets;
packets += 1;
}
if !packets return false, "the recording holds no frames", 0;
result = av_write_trailer(output);
if result < 0 return false, av_error("FFmpeg cannot finish the MP4", result), packets;
return true, "", packets;
}

View File

@ -308,6 +308,24 @@ IID_DIRECT3D_DEVICE := #run guid("A37624AB-8D5F-4650-9D3E-9EAE3D9BC670");
IID_DXGI_INTERFACE_ACCESS := #run guid("A9B3D012-3DF2-4EE3-B8D1-8695F457D3C1"); IID_DXGI_INTERFACE_ACCESS := #run guid("A9B3D012-3DF2-4EE3-B8D1-8695F457D3C1");
IID_CLOSABLE := #run guid("30D5A829-7FA4-4026-83BB-D75BAE4EA99E"); IID_CLOSABLE := #run guid("30D5A829-7FA4-4026-83BB-D75BAE4EA99E");
guid :: (text: string) -> GUID {
hex :: (s: string) -> u64 {
value: u64;
for 0..s.count - 1 {
c := s[it];
digit: u64 = ifx c >= #char "a" then cast(u64) (c - #char "a" + 10) else ifx c >= #char "A" then cast(u64) (c - #char "A" + 10) else cast(u64) (c - #char "0");
value = value * 16 + digit;
}
return value;
}
g: GUID;
g.Data1 = cast(u32) hex(slice(text, 0, 8));
g.Data2 = cast(u16) hex(slice(text, 9, 4));
g.Data3 = cast(u16) hex(slice(text, 14, 4));
for 0..1 g.Data4[it] = cast(u8) hex(slice(text, 19 + it * 2, 2));
for 0..5 g.Data4[it + 2] = cast(u8) hex(slice(text, 24 + it * 2, 2));
return g;
}
// IUnknown, then IInspectable's three: every WinRT interface starts so. // IUnknown, then IInspectable's three: every WinRT interface starts so.
Inspectable_Methods :: struct { Inspectable_Methods :: struct {

View File

@ -23,7 +23,6 @@ Screen_Capture_Job :: struct {
source: string; // how the screen was taken: Record_Stats.source source: string; // how the screen was taken: Record_Stats.source
dropped: s64; // frames the encoder fell too far behind for dropped: s64; // frames the encoder fell too far behind for
paused_seconds: float64; // left out of it paused_seconds: float64; // left out of it
audio: string; // the sound's sources, "Speakers (...); Microphone (...)", or ""
thread: Thread; thread: Thread;
busy: bool; // cleared by the thread as it ends busy: bool; // cleared by the thread as it ends
@ -143,7 +142,6 @@ screen_capture_job_free :: (job: *Screen_Capture_Job) {
free(job.recording); free(job.recording);
free(job.encoder); free(job.encoder);
free(job.source); free(job.source);
free(job.audio);
free(job); free(job);
} }

View File

@ -9,10 +9,8 @@
// //
// While it records, a red border is drawn just outside the region and a toolbar beside it gives the // While it records, a red border is drawn just outside the region and a toolbar beside it gives the
// time, Pause, Stop and Discard (app/screen_record_win32.jai); both are kept out of the recording. // time, Pause, Stop and Discard (app/screen_record_win32.jai); both are kept out of the recording.
// Time paused is left out of the video. With sound: what the computer plays and the microphone, as // Time paused is left out of the video. The frame rate, the cursor and how the screen is taken are
// [video] in the settings file says, as are the frame rate, the cursor and how the screen is taken. // [video] in the settings file.
// The toolbar's System sound and Mic buttons turn each on and off while recording, and are kept in
// [video] for the next: every recording has a sound track for them, silent while nothing is on.
// //
// A recording that stops by itself (the screen went away, the encoder failed) keeps what it has, and // A recording that stops by itself (the screen went away, the encoder failed) keeps what it has, and
// so does quitting while recording. One left in the temporary folder by a crash is offered at the next // so does quitting while recording. One left in the temporary folder by a crash is offered at the next
@ -52,25 +50,7 @@ screen_record_frame :: () {
screen_record_stop(); screen_record_stop();
return; return;
} }
for kind: Audio_Source_Kind.[.SYSTEM, .MICROPHONE] { if record_ui_take_ask() == {
failed, why := record_audio_failed(*screen_record.recorder, kind);
if !failed continue;
toast_show(.Error, "% could not be turned on: %", screen_record_audio_name(kind), why);
screen_record_remember_audio(kind, false);
record_ui_refresh();
}
ask := record_ui_take_ask();
if ask == .SYSTEM_SOUND || ask == .MICROPHONE {
kind: Audio_Source_Kind = ifx ask == .SYSTEM_SOUND then .SYSTEM else .MICROPHONE;
on := !record_audio_on(*screen_record.recorder, kind);
if record_set_audio(*screen_record.recorder, kind, on) {
log("[record] % turned %.", screen_record_audio_name(kind), ifx on then "on" else "off");
toast_show(.Success, "% %", screen_record_audio_name(kind), ifx on then "on" else "off");
screen_record_remember_audio(kind, on);
}
record_ui_refresh();
}
if ask == {
case .PAUSE; case .PAUSE;
pause := !record_paused(*screen_record.recorder); pause := !record_paused(*screen_record.recorder);
record_pause(*screen_record.recorder, pause); record_pause(*screen_record.recorder, pause);
@ -224,9 +204,6 @@ screen_record_start :: (immediate := false) {
options.region = .{shot.left, shot.top, shot.left + shot.width, shot.top + shot.height}; options.region = .{shot.left, shot.top, shot.left + shot.width, shot.top + shot.height};
options.fps = cast(s32) app_user_settings.video_fps; options.fps = cast(s32) app_user_settings.video_fps;
options.cursor = app_user_settings.video_cursor; options.cursor = app_user_settings.video_cursor;
options.system_audio = app_user_settings.video_system_audio;
options.microphone = app_user_settings.video_microphone;
options.audio_track = true; // so the toolbar can turn a source on mid-recording
if app_user_settings.video_capture == { if app_user_settings.video_capture == {
case "dxgi"; options.capture = .DXGI; case "dxgi"; options.capture = .DXGI;
case "wgc"; options.capture = .WGC; case "wgc"; options.capture = .WGC;
@ -247,28 +224,10 @@ screen_record_start :: (immediate := false) {
if stats.fell_back_from log("[record] Desktop Duplication could not run: %", stats.fell_back_from); if stats.fell_back_from log("[record] Desktop Duplication could not run: %", stats.fell_back_from);
log("[record] Recording % x % at %, % (%) of \"%\" (%) with % and %, to %.", stats.width, stats.height, shot.left, shot.top, log("[record] Recording % x % at %, % (%) of \"%\" (%) with % and %, to %.", stats.width, stats.height, shot.left, shot.top,
ifx shot.whole_screen then "the whole screen" else "a region", shot.window_title, shot.app_name, stats.source, stats.encoder, path); ifx shot.whole_screen then "the whole screen" else "a region", shot.window_title, shot.app_name, stats.source, stats.encoder, path);
if stats.audio log("[record] Sound: %.", stats.audio); toast_show(.Success, "Recording % x %: % stops", stats.width, stats.height, screen_record_hotkey_text());
if stats.audio_errors {
log_error("[record] Sound not recorded: %.", stats.audio_errors);
toast_show(.Error, "Recording without some sound: %", stats.audio_errors);
} else {
toast_show(.Success, "Recording % x %: % stops", stats.width, stats.height, screen_record_hotkey_text());
}
record_ui_open(options.region, shot.whole_screen, stats.width, stats.height); record_ui_open(options.region, shot.whole_screen, stats.width, stats.height);
} }
// A sound button's state, into [video] for the next recording.
screen_record_remember_audio :: (kind: Audio_Source_Kind, on: bool) {
setting := ifx kind == .SYSTEM then *app_user_settings.video_system_audio else *app_user_settings.video_microphone;
if setting.* == on return;
setting.* = on;
app_save_user_settings_file();
}
screen_record_audio_name :: (kind: Audio_Source_Kind) -> string {
return ifx kind == .SYSTEM then "System sound" else "The microphone";
}
// Stops, and deletes what was recorded (the toolbar's Discard, which asked first). // Stops, and deletes what was recorded (the toolbar's Discard, which asked first).
screen_record_discard :: () { screen_record_discard :: () {
if !screen_record.recording return; if !screen_record.recording return;
@ -319,7 +278,6 @@ screen_record_stop :: (shutdown := false) {
job.source = copy_string(stats.source,, context.default_allocator); job.source = copy_string(stats.source,, context.default_allocator);
job.dropped = stats.dropped; job.dropped = stats.dropped;
job.paused_seconds = stats.paused_seconds; job.paused_seconds = stats.paused_seconds;
job.audio = copy_string(stats.audio,, context.default_allocator);
screen_record.shot = .{}; screen_record.shot = .{};
screen_record.date, screen_record.path = "", ""; screen_record.date, screen_record.path = "", "";
screen_capture_job_start(job); screen_capture_job_start(job);
@ -327,34 +285,21 @@ screen_record_stop :: (shutdown := false) {
// The recording's .mcap in the library, from the job's MP4, which is deleted once it is in: its path, // The recording's .mcap in the library, from the job's MP4, which is deleted once it is in: its path,
// or why it could not be (temp). On the job's thread. // or why it could not be (temp). On the job's thread.
//
// The MP4 was written fragmented, so a crash leaves one that plays; it is made a regular MP4 first
// (record_remux), with its length and index in its header, as players need to show its time and seek
// in it. Should that fail, the MP4 goes in as it was.
screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string { screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string {
_, recorded_size, found := file_modtime_and_size(job.recording); _, size, found := file_modtime_and_size(job.recording);
if !found || recorded_size <= 0 return "", tprint("% is missing", job.recording); if !found || size <= 0 return "", tprint("% is missing", job.recording);
mp4 := job.recording; image, video, decoded, decode_error := video_poster(job.recording);
remuxed := tprint("%.regular.mp4", path_strip_extension(job.recording));
remux_ok, remux_error, packets := record_remux(job.recording, remuxed);
if remux_ok mp4 = remuxed;
else { log_error("[record] % stays fragmented: %", job.recording, remux_error); file_delete(remuxed); }
defer if remux_ok file_delete(remuxed);
_, size, _ := file_modtime_and_size(mp4);
image, video, decoded, decode_error := video_poster(mp4);
if !decoded return "", tprint("the recording cannot be read back: %", decode_error); if !decoded return "", tprint("the recording cannot be read back: %", decode_error);
defer image_free(*image); defer image_free(*image);
preview, preview_width, preview_height, preview_ok, preview_error := image_preview_png(image, MCAP_PREVIEW_MAX_SIDE); preview, preview_width, preview_height, preview_ok, preview_error := image_preview_png(image, MCAP_PREVIEW_MAX_SIDE);
if !preview_ok return "", preview_error; if !preview_ok return "", preview_error;
poster, _, _, poster_ok, poster_error := image_preview_png(image, max(image.width, image.height)); // at its own size poster, _, _, poster_ok, poster_error := image_preview_png(image, max(image.width, image.height)); // at its own size
if !poster_ok return "", poster_error; if !poster_ok return "", poster_error;
hash, hashed := import_hash_file(.{path = mp4, offset = 0, size = size}); hash, hashed := import_hash_file(.{path = job.recording, offset = 0, size = size});
if !hashed return "", tprint("% cannot be read", mp4); if !hashed return "", tprint("% cannot be read", job.recording);
shot := job.shot; shot := job.shot;
known := shot.width > 0; // a crash's leftover knows no region, window or encoder if !shot.width { shot.width, shot.height = image.width, image.height; } // a crash's leftover: no more is known
if !known { shot.width, shot.height = image.width, image.height; }
doc := New(Capture_Document,, temp); doc := New(Capture_Document,, temp);
capture_document_init(doc, temp); capture_document_init(doc, temp);
doc.comment = tprint("Recorded by % %.", APP_NAME, VERSION); doc.comment = tprint("Recorded by % %.", APP_NAME, VERSION);
@ -363,11 +308,9 @@ screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string
import_add(doc, metadata, "CaptureDate", job.date); import_add(doc, metadata, "CaptureDate", job.date);
import_add(doc, metadata, "WindowName", import_text(shot.window_title)); import_add(doc, metadata, "WindowName", import_text(shot.window_title));
info := capture_add_file(doc, "capture.json"); info := capture_add_file(doc, "capture.json");
if known { import_add(doc, info, "Region", ifx shot.whole_screen then "screen" else "selection");
import_add(doc, info, "Region", ifx shot.whole_screen then "screen" else "selection"); import_add_number(doc, info, "Left", xx shot.left);
import_add_number(doc, info, "Left", xx shot.left); import_add_number(doc, info, "Top", xx shot.top);
import_add_number(doc, info, "Top", xx shot.top);
}
import_add_number(doc, info, "Width", xx shot.width); import_add_number(doc, info, "Width", xx shot.width);
import_add_number(doc, info, "Height", xx shot.height); import_add_number(doc, info, "Height", xx shot.height);
if shot.app_path import_add(doc, info, "AppPath", import_text(shot.app_path)); if shot.app_path import_add(doc, info, "AppPath", import_text(shot.app_path));
@ -380,8 +323,6 @@ screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string
if job.encoder import_add(doc, info, "Encoder", import_text(job.encoder)); if job.encoder import_add(doc, info, "Encoder", import_text(job.encoder));
if job.source import_add(doc, info, "Capture", import_text(job.source)); if job.source import_add(doc, info, "Capture", import_text(job.source));
import_add_number(doc, info, "DroppedFrames", xx job.dropped); import_add_number(doc, info, "DroppedFrames", xx job.dropped);
capture_object_add(doc, *info.root, "Audio", capture_boolean(video.has_audio));
if job.audio import_add(doc, info, "AudioSources", import_text(job.audio));
if job.paused_seconds > 0 import_add_number(doc, info, "Paused", import_milliseconds(job.paused_seconds)); if job.paused_seconds > 0 import_add_number(doc, info, "Paused", import_milliseconds(job.paused_seconds));
name := import_out_name(job.date, hash); name := import_out_name(job.date, hash);
@ -396,7 +337,7 @@ screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string
m.rendered = poster; m.rendered = poster;
path, folder_error := screen_capture_out_path(name); path, folder_error := screen_capture_out_path(name);
if folder_error return "", folder_error; if folder_error return "", folder_error;
written, write_error := mcap_write_streamed(path, m, .{path = mp4, offset = 0, size = size}); written, write_error := mcap_write_streamed(path, m, .{path = job.recording, offset = 0, size = size});
if !written return "", write_error; if !written return "", write_error;
file_delete(job.recording); file_delete(job.recording);
return path, ""; return path, "";

View File

@ -1,6 +1,5 @@
// While recording (app/screen_record.jai), on Windows: a thin red border just outside the region, and a // While recording (app/screen_record.jai), on Windows: a thin red border just outside the region, and a
// toolbar beside it with the time, the size, frames dropped, and System sound, Mic, Pause, Stop and // toolbar beside it with the time, the size, frames dropped, and Pause, Stop and Discard. Both are
// Discard. Both are
// plain GDI windows, as the overlay is, so they work while mcap is hidden in the tray, and both are // plain GDI windows, as the overlay is, so they work while mcap is hidden in the tray, and both are
// kept out of every capture (WDA_EXCLUDEFROMCAPTURE), the recording's own included. The border is // kept out of every capture (WDA_EXCLUDEFROMCAPTURE), the recording's own included. The border is
// outside the region anyway, so it could never be in the video. // outside the region anyway, so it could never be in the video.
@ -10,7 +9,7 @@
// message, as Stop and Discard close the toolbar. Discard asks first, in the toolbar itself: a // message, as Stop and Discard close the toolbar. Discard asks first, in the toolbar itself: a
// message box would be in the recording. // message box would be in the recording.
Record_UI_Ask :: enum u8 { NONE; SYSTEM_SOUND; MICROPHONE; PAUSE; STOP; DISCARD; } Record_UI_Ask :: enum u8 { NONE; PAUSE; STOP; DISCARD; }
record_ui: struct { record_ui: struct {
border, toolbar: HWND; border, toolbar: HWND;
@ -20,7 +19,7 @@ record_ui: struct {
asked: Record_UI_Ask; // a button pressed, for screen_record_frame asked: Record_UI_Ask; // a button pressed, for screen_record_frame
confirming: bool; // Discard pressed once: "Discard it? Yes No" confirming: bool; // Discard pressed once: "Discard it? Yes No"
hovered: s32 = -1; // the button under the mouse hovered: s32 = -1; // the button under the mouse
buttons: [5] Record_UI_Button; buttons: [3] Record_UI_Button;
button_count: s32; button_count: s32;
started: Apollo_Time; started: Apollo_Time;
size_text: string; // "1920 × 1080"; heap size_text: string; // "1920 × 1080"; heap
@ -171,13 +170,10 @@ record_toolbar_buttons :: () -> [] Record_UI_Button {
record_ui.buttons[1] = .{label = "No", confirm = true}; record_ui.buttons[1] = .{label = "No", confirm = true};
record_ui.button_count = 2; record_ui.button_count = 2;
} else { } else {
recorder := *screen_record.recorder; record_ui.buttons[0] = .{label = ifx record_paused(*screen_record.recorder) then "Resume" else "Pause", ask = .PAUSE};
record_ui.buttons[0] = .{label = ifx record_audio_on(recorder, .SYSTEM) then "System sound: on" else "System sound: off", ask = .SYSTEM_SOUND}; record_ui.buttons[1] = .{label = "Stop", ask = .STOP};
record_ui.buttons[1] = .{label = ifx record_audio_on(recorder, .MICROPHONE) then "Mic: on" else "Mic: off", ask = .MICROPHONE}; record_ui.buttons[2] = .{label = "Discard", confirm = true};
record_ui.buttons[2] = .{label = ifx record_paused(recorder) then "Resume" else "Pause", ask = .PAUSE}; record_ui.button_count = 3;
record_ui.buttons[3] = .{label = "Stop", ask = .STOP};
record_ui.buttons[4] = .{label = "Discard", confirm = true};
record_ui.button_count = 5;
} }
return array_view(record_ui.buttons, 0, record_ui.button_count); return array_view(record_ui.buttons, 0, record_ui.button_count);
} }
@ -200,7 +196,7 @@ record_toolbar_size :: () -> width: s32, height: s32 {
padding, gap := record_ui_px(10), record_ui_px(8); padding, gap := record_ui_px(10), record_ui_px(8);
status_width := text_width(dc, tprint("PAUSED 00:00:00 %", record_ui.size_text)); status_width := text_width(dc, tprint("PAUSED 00:00:00 %", record_ui.size_text));
dropped_width := text_width(dc, "999 dropped, screen lost 9 times"); dropped_width := text_width(dc, "999 dropped, screen lost 9 times");
buttons_width := text_width(dc, "System sound: off") + text_width(dc, "Mic: off") + text_width(dc, "Resume") + text_width(dc, "Stop") + text_width(dc, "Yes, discard") + 5 * 2 * padding + 4 * gap; buttons_width := text_width(dc, "Resume") + text_width(dc, "Stop") + text_width(dc, "Yes, discard") + 3 * 2 * padding + 2 * gap;
height := record_ui_px(34); height := record_ui_px(34);
width := padding + record_ui_px(12) + gap + status_width + gap + dropped_width + gap + buttons_width + padding; width := padding + record_ui_px(12) + gap + status_width + gap + dropped_width + gap + buttons_width + padding;
return width, height; return width, height;
@ -317,9 +313,7 @@ record_toolbar_paint :: (hwnd: HWND) {
label_width := text_width(dc, it.label); label_width := text_width(dc, it.label);
r := RECT.{right - label_width - 2 * padding, (height - button_height) / 2, right, (height - button_height) / 2 + button_height}; r := RECT.{right - label_width - 2 * padding, (height - button_height) / 2, right, (height - button_height) / 2 + button_height};
record_ui.buttons[it_index].rect = r; record_ui.buttons[it_index].rect = r;
// A sound that is on, in the record red. fill(r, ifx it_index == record_ui.hovered then cast(u32) RECORD_UI_HOVER else cast(u32) RECORD_UI_BUTTON);
lit := (it.ask == .SYSTEM_SOUND && record_audio_on(*screen_record.recorder, .SYSTEM)) || (it.ask == .MICROPHONE && record_audio_on(*screen_record.recorder, .MICROPHONE));
fill(r, ifx lit then cast(u32) RECORD_UI_RED else ifx it_index == record_ui.hovered then cast(u32) RECORD_UI_HOVER else cast(u32) RECORD_UI_BUTTON);
SetTextColor(dc, ifx it.ask == .DISCARD then cast(u32) RECORD_UI_AMBER else cast(u32) RECORD_UI_TEXT); SetTextColor(dc, ifx it.ask == .DISCARD then cast(u32) RECORD_UI_AMBER else cast(u32) RECORD_UI_TEXT);
wide := utf8_to_wide(it.label,, temp); wide := utf8_to_wide(it.label,, temp);
count: s32; count: s32;

View File

@ -732,10 +732,7 @@ screen_record_tests :: () {
m, parsed := mcap_parse(cast([] u8) read_entire_file(path,, temp)); m, parsed := mcap_parse(cast([] u8) read_entire_file(path,, temp));
expect(parsed, "it reads back"); expect(parsed, "it reads back");
if parsed { if parsed {
// Recorded fragmented, it goes in a regular MP4: the same frames, its header first. expect(m.original.count == mp4_size && ends_with(m.original_name, ".mp4"), "the MP4 is its original: % of % bytes, %", m.original.count, mp4_size, m.original_name);
original := cast(string) m.original;
expect(ends_with(m.original_name, ".mp4") && slice(original, 4, 4) == "ftyp" && find_index_from_left(original, "moof") < 0 && find_index_from_left(original, "moov") < find_index_from_left(original, "mdat"),
"the MP4, made regular, is its original: % (% bytes, % recorded)", m.original_name, m.original.count, mp4_size);
expect(m.rendered_kind == .OWN, "a frame of it is its rendered image"); expect(m.rendered_kind == .OWN, "a frame of it is its rendered image");
poster, decoded := image_decode(m.rendered); poster, decoded := image_decode(m.rendered);
expect(decoded && poster.width == WIDTH && poster.height == HEIGHT, "at the recording's size"); expect(decoded && poster.width == WIDTH && poster.height == HEIGHT, "at the recording's size");
@ -782,67 +779,6 @@ screen_record_tests :: () {
} }
file_delete(leftover_path); file_delete(leftover_path);
} }
// With sound: an AAC stream beside the video, kept through the remux, and capture.json says so.
encoder = .{};
opened, open_error = encoder_open(*encoder, mp4, WIDTH, HEIGHT, 30, "", audio = true);
expect(opened && encoder.audio != null, "an encoder with sound opened: %", open_error);
if opened {
tone := NewArray(encoder.audio_frame_size * 2, float,, temp);
for frame: 0..29 encoder_write(*encoder, pixels.data, frame * 3000);
for block: 0..46 { // about a second of 440 Hz
for i: 0..encoder.audio_frame_size - 1 {
v := sin(cast(float) (block * encoder.audio_frame_size + i) * 2 * PI * 440 / 48000) * 0.5;
tone[i * 2], tone[i * 2 + 1] = v, v;
}
encoder_write_audio(*encoder, tone.data, block * encoder.audio_frame_size);
}
encoder_close(*encoder);
sounding: Screen_Capture_Job;
sounding.video = true;
sounding.recording = mp4;
sounding.date = "2026-10-10 08:30:00";
sounding.audio = "Speakers (test)";
sounding.shot.width, sounding.shot.height = WIDTH, HEIGHT;
sounding_path, sounding_error := screen_record_write(*sounding);
expect(!sounding_error, "a recording with sound is written: %", sounding_error);
if !sounding_error {
m4, parsed4 := mcap_parse(cast([] u8) read_entire_file(sounding_path,, temp));
expect(parsed4 && find_index_from_left(cast(string) m4.original, "mp4a") >= 0, "its MP4 has the AAC stream (mp4a)");
expect(parsed4 && contains(m4.text, "Speakers (test)"), "capture.json names the sound's source");
file_delete(sounding_path);
}
}
// One a crash cut short: a frame every 100 ms for 5 s makes fragments at 0, 2 and 4 s (a
// keyframe each 2 s, however few the frames); cut at the last, as a crash would, it goes into the
// library as a regular MP4 (header first, no fragments) of the 4 s before the cut.
encoder = .{};
encoder_open(*encoder, mp4, WIDTH, HEIGHT, 30, "");
for frame: 0..49 encoder_write(*encoder, pixels.data, frame * 9000);
encoder_close(*encoder);
whole := read_entire_file(mp4,, temp);
last_fragment := -1;
for 4..whole.count - 4 if whole[it] == #char "m" && slice(whole, it, 4) == "moof" last_fragment = it - 4;
expect(last_fragment > 0, "the recording is fragmented");
write_entire_file(mp4, slice(whole, 0, last_fragment));
crashed: Screen_Capture_Job;
crashed.video = true;
crashed.recording = mp4;
crashed.date = "2026-10-10 07:48:52";
crashed_path, crashed_error := screen_record_write(*crashed);
expect(!crashed_error, "a cut recording is written: %", crashed_error);
if !crashed_error {
m3, parsed3 := mcap_parse(cast([] u8) read_entire_file(crashed_path,, temp));
original := cast(string) m3.original;
moov := find_index_from_left(original, "moov");
mdat := find_index_from_left(original, "mdat");
expect(parsed3 && moov > 0 && moov < mdat && find_index_from_left(original, "moof") < 0, "a regular MP4, its header first");
crashed_entry := catalog_load(crashed_path);
expect(crashed_entry && crashed_entry.duration > 3.5 && crashed_entry.duration < 4.5, "the 4 s before the cut: %", ifx crashed_entry then crashed_entry.duration else -1);
if crashed_entry catalog_entry_free(crashed_entry);
file_delete(crashed_path);
}
} }
} }

View File

@ -311,8 +311,6 @@ App_User_Settings :: struct {
video_fps: s64 = VIDEO_FPS_DEFAULT; video_fps: s64 = VIDEO_FPS_DEFAULT;
video_cursor := true; video_cursor := true;
video_capture: string; video_capture: string;
video_system_audio := true; // what the computer plays
video_microphone: bool; // off unless asked for
// The [hotkeys] lines. Only meaningful when the section was seen; see hotkeys_install. // The [hotkeys] lines. Only meaningful when the section was seen; see hotkeys_install.
hotkey_bindings: [..] Hotkey_Binding; hotkey_bindings: [..] Hotkey_Binding;
@ -382,12 +380,12 @@ app_load_user_settings_file :: () -> write_back: bool {
if hotkeys_missing.count || global_missing.count return true; if hotkeys_missing.count || global_missing.count return true;
#if OS == .LINUX seen.global_hotkeys = true; // not written there: Linux has no global hotkeys yet #if OS == .LINUX seen.global_hotkeys = true; // not written there: Linux has no global hotkeys yet
return !(seen.theme && seen.width && seen.height && seen.font_size && seen.font_scale && seen.event_log && seen.hotkeys && seen.global_hotkeys && seen.ocr_model && seen.video_fps && seen.video_cursor && seen.video_capture && seen.video_system_audio && seen.video_microphone); return !(seen.theme && seen.width && seen.height && seen.font_size && seen.font_scale && seen.event_log && seen.hotkeys && seen.global_hotkeys && seen.ocr_model && seen.video_fps && seen.video_cursor && seen.video_capture);
} }
Settings_Seen :: struct { Settings_Seen :: struct {
theme, width, height, font_size, font_scale, event_log, hotkeys, global_hotkeys, ocr_model: bool; theme, width, height, font_size, font_scale, event_log, hotkeys, global_hotkeys, ocr_model: bool;
video_fps, video_cursor, video_capture, video_system_audio, video_microphone: bool; video_fps, video_cursor, video_capture: bool;
} }
// Of the file's bytes as last loaded or written, so a watcher event for our own save, or for an // Of the file's bytes as last loaded or written, so a watcher event for our own save, or for an
@ -522,10 +520,6 @@ settings_parse :: (path: string, file_data: string, out: *App_User_Settings) ->
seen.video_fps = settings_read_int(*out.video_fps, path, line_number, key, value, VIDEO_FPS_MIN, VIDEO_FPS_MAX); seen.video_fps = settings_read_int(*out.video_fps, path, line_number, key, value, VIDEO_FPS_MIN, VIDEO_FPS_MAX);
} else if section == "[video]" && key == "cursor" { } else if section == "[video]" && key == "cursor" {
seen.video_cursor = settings_read_bool(*out.video_cursor, path, line_number, key, value); seen.video_cursor = settings_read_bool(*out.video_cursor, path, line_number, key, value);
} else if section == "[video]" && key == "system_audio" {
seen.video_system_audio = settings_read_bool(*out.video_system_audio, path, line_number, key, value);
} else if section == "[video]" && key == "microphone" {
seen.video_microphone = settings_read_bool(*out.video_microphone, path, line_number, key, value);
} else if section == "[video]" && key == "capture" { } else if section == "[video]" && key == "capture" {
capture := to_lower_copy(settings_unquote(value),, temp); capture := to_lower_copy(settings_unquote(value),, temp);
if capture == "auto" || capture == "dxgi" || capture == "wgc" { if capture == "auto" || capture == "dxgi" || capture == "wgc" {
@ -670,7 +664,6 @@ app_save_user_settings_file :: () -> bool {
if seen.event_log app_user_settings.event_log_capacity = on_disk.event_log_capacity; if seen.event_log app_user_settings.event_log_capacity = on_disk.event_log_capacity;
if seen.video_fps app_user_settings.video_fps = on_disk.video_fps; if seen.video_fps app_user_settings.video_fps = on_disk.video_fps;
if seen.video_cursor app_user_settings.video_cursor = on_disk.video_cursor; if seen.video_cursor app_user_settings.video_cursor = on_disk.video_cursor;
// video_system_audio and video_microphone are the app's: the recording toolbar's buttons set them.
if seen.video_capture { if seen.video_capture {
free(app_user_settings.video_capture,, context.default_allocator); free(app_user_settings.video_capture,, context.default_allocator);
app_user_settings.video_capture = copy_string(on_disk.video_capture,, context.default_allocator); app_user_settings.video_capture = copy_string(on_disk.video_capture,, context.default_allocator);
@ -698,13 +691,10 @@ app_save_user_settings_file :: () -> bool {
append(*builder, "[video]\n"); append(*builder, "[video]\n");
append(*builder, "# Screen recording (Ctrl+PrintScreen; Windows). fps: frames a second at most, written only when the\n"); append(*builder, "# Screen recording (Ctrl+PrintScreen; Windows). fps: frames a second at most, written only when the\n");
append(*builder, "# screen changes. cursor: true or false. capture: auto (Desktop Duplication, else\n"); append(*builder, "# screen changes. cursor: true or false. capture: auto (Desktop Duplication, else\n");
append(*builder, "# Windows.Graphics.Capture), dxgi or wgc. system_audio: what the computer plays; microphone: the\n"); append(*builder, "# Windows.Graphics.Capture), dxgi or wgc.\n");
append(*builder, "# default input device. Each true or false.\n");
print_to_builder(*builder, "fps: %\n", app_user_settings.video_fps); print_to_builder(*builder, "fps: %\n", app_user_settings.video_fps);
print_to_builder(*builder, "cursor: %\n", ifx app_user_settings.video_cursor then "true" else "false"); print_to_builder(*builder, "cursor: %\n", ifx app_user_settings.video_cursor then "true" else "false");
print_to_builder(*builder, "capture: %\n", ifx app_user_settings.video_capture.count then app_user_settings.video_capture else "auto"); print_to_builder(*builder, "capture: %\n\n", ifx app_user_settings.video_capture.count then app_user_settings.video_capture else "auto");
print_to_builder(*builder, "system_audio: %\n", ifx app_user_settings.video_system_audio then "true" else "false");
print_to_builder(*builder, "microphone: %\n\n", ifx app_user_settings.video_microphone then "true" else "false");
append(*builder, "[logging]\n"); append(*builder, "[logging]\n");
append(*builder, "# Entries the Event Log keeps in memory. Past it the oldest are dropped; the log file keeps every line.\n"); append(*builder, "# Entries the Event Log keeps in memory. Past it the oldest are dropped; the log file keeps every line.\n");
print_to_builder(*builder, "event_log_capacity: %\n", app_user_settings.event_log_capacity); print_to_builder(*builder, "event_log_capacity: %\n", app_user_settings.event_log_capacity);

View File

@ -20,12 +20,6 @@
// WDA_EXCLUDEFROMCAPTURE: is it out of the recording, and the other in? // WDA_EXCLUDEFROMCAPTURE: is it out of the recording, and the other in?
// -animate moves a square across the region the whole time, so the screen changes every // -animate moves a square across the region the whole time, so the screen changes every
// frame: for measuring throughput. // frame: for measuring throughput.
// -remux IN OUT a recording's fragmented MP4 made a regular one, as the app does, and nothing else.
// -audio WHICH records sound too: system (what the computer plays), mic, or both.
// -mic-toggle the microphone turned on 3 s in, and off at 7 s, as the toolbar does.
// -system-toggle likewise for what the computer plays.
// -sync-test every 2 s a white square flashes on the region and a 1 kHz beep plays; then the
// flashes and beeps are found in the file and their offsets given, and how they drift.
// -check FILE reads FILE back with FFmpeg and says what it holds, and nothing else: for a // -check FILE reads FILE back with FFmpeg and says what it holds, and nothing else: for a
// recording cut short by a crash. // recording cut short by a crash.
// -snapshot DIR the last frame captured (captured.png) and a frame decoded back from the file // -snapshot DIR the last frame captured (captured.png) and a frame decoded back from the file
@ -46,9 +40,6 @@ main :: () {
list_only, cuda, exclude_test, dpi_unaware, animate, border_test, cursor_test := false, false, false, false, false, false, false; list_only, cuda, exclude_test, dpi_unaware, animate, border_test, cursor_test := false, false, false, false, false, false, false;
snapshot_folder: string; snapshot_folder: string;
pause_seconds: float64; pause_seconds: float64;
sync_test := false;
mic_toggle := false;
system_toggle := false;
i := 1; i := 1;
number :: (index: s64) -> s32 #expand { return cast(s32) string_to_int(`args[index]); } number :: (index: s64) -> s32 #expand { return cast(s32) string_to_int(`args[index]); }
while i < args.count { while i < args.count {
@ -77,16 +68,6 @@ main :: () {
if a == "-simulate-unsupported" { options.testing |= .DXGI_UNSUPPORTED; i += 1; continue; } if a == "-simulate-unsupported" { options.testing |= .DXGI_UNSUPPORTED; i += 1; continue; }
if a == "-poll" { options.testing |= .WGC_POLL; i += 1; continue; } if a == "-poll" { options.testing |= .WGC_POLL; i += 1; continue; }
if a == "-lose" { options.testing |= .LOSE_DUPLICATION; i += 1; continue; } if a == "-lose" { options.testing |= .LOSE_DUPLICATION; i += 1; continue; }
if a == "-sync-test" { sync_test = true; i += 1; continue; }
if a == "-mic-toggle" { mic_toggle = true; options.audio_track = true; i += 1; continue; }
if a == "-system-toggle" { system_toggle = true; options.audio_track = true; i += 1; continue; }
if a == "-audio" && more >= 1 {
which := args[i + 1];
options.system_audio = which == "system" || which == "both";
options.microphone = which == "mic" || which == "both";
if !options.system_audio && !options.microphone { print("-audio takes system, mic or both\n"); exit(2); }
i += 2; continue;
}
if a == "-pause" && more >= 1 { pause_seconds = string_to_float64(args[i + 1]); i += 2; continue; } if a == "-pause" && more >= 1 { pause_seconds = string_to_float64(args[i + 1]); i += 2; continue; }
if a == "-cursor-test" { cursor_test = true; i += 1; continue; } if a == "-cursor-test" { cursor_test = true; i += 1; continue; }
if a == "-capture" && more >= 1 { if a == "-capture" && more >= 1 {
@ -97,12 +78,6 @@ main :: () {
else { print("-capture takes auto, dxgi or wgc\n"); exit(2); } else { print("-capture takes auto, dxgi or wgc\n"); exit(2); }
i += 2; continue; i += 2; continue;
} }
if a == "-remux" && more >= 2 {
ok, error, packets := record_remux(args[i + 1], args[i + 2]);
if !ok { print("Cannot remux %: %\n", args[i + 1], error); exit(1); }
print("% packets into %\n", packets, args[i + 2]);
exit(0);
}
if a == "-check" && more >= 1 { if a == "-check" && more >= 1 {
image, info, ok, error := video_poster(args[i + 1]); image, info, ok, error := video_poster(args[i + 1]);
if !ok { print("%: cannot be read: %\n", args[i + 1], error); exit(1); } if !ok { print("%: cannot be read: %\n", args[i + 1], error); exit(1); }
@ -167,15 +142,6 @@ main :: () {
cursor_at = .{region.left + (region.right - region.left) / 2, region.top + (region.bottom - region.top) / 2}; cursor_at = .{region.left + (region.right - region.left) / 2, region.top + (region.bottom - region.top) / 2};
SetCursorPos(cursor_at.x, cursor_at.y); SetCursorPos(cursor_at.x, cursor_at.y);
} }
flash: HWND;
beep: [] u8;
next_flash: float64 = 1;
if sync_test {
beep = beep_wav();
side := min(cast(s32) 200, (region.right - region.left) / 3, (region.bottom - region.top) / 3);
flash = square_window("record-probe flash", 0x00FFFFFF, region.left + (region.right - region.left - side) / 2, region.top + (region.bottom - region.top - side) / 2, side);
ShowWindow(flash, SW_HIDE);
}
mover: HWND; mover: HWND;
if animate mover = square_window("record-probe animate", 0x0000A5FF, region.left, region.top, 120); if animate mover = square_window("record-probe animate", 0x0000A5FF, region.left, region.top, 120);
@ -200,27 +166,6 @@ main :: () {
print(" % s: % captured, % sent (% repeats), % written, % dropped, % lost, % KB\n", print(" % s: % captured, % sent (% repeats), % written, % dropped, % lost, % KB\n",
FF(to_float64_seconds(last_print - start), 0), s.captured, s.emitted, s.repeats, s.written, s.dropped, s.lost, s.bytes / 1024); FF(to_float64_seconds(last_print - start), 0), s.captured, s.emitted, s.repeats, s.written, s.dropped, s.lost, s.bytes / 1024);
} }
for kind: Audio_Source_Kind.[.SYSTEM, .MICROPHONE] {
if !(ifx kind == .SYSTEM then system_toggle else mic_toggle) continue;
elapsed := to_float64_seconds(current_time_monotonic() - start);
should := elapsed >= 3 && elapsed < 7;
if should != record_audio_on(*recorder, kind) && (elapsed < 3.5 || (elapsed >= 7 && elapsed < 7.5)) {
record_set_audio(*recorder, kind, should);
print(" % s: % %\n", FF(elapsed, 1), kind, ifx should then "on" else "off");
}
failed, why := record_audio_failed(*recorder, kind);
if failed print(" % failed: %\n", kind, why);
}
if flash { // a flash and a beep together every 2 s, the flash for 200 ms
elapsed := to_float64_seconds(current_time_monotonic() - start);
if elapsed >= next_flash {
ShowWindow(flash, SW_SHOWNOACTIVATE);
PlaySoundW(beep.data, null, SND_MEMORY | SND_ASYNC);
next_flash += 2;
} else if elapsed >= next_flash - 2 + 0.2 {
ShowWindow(flash, SW_HIDE);
}
}
if pause_seconds > 0 { if pause_seconds > 0 {
elapsed := to_float64_seconds(current_time_monotonic() - start); elapsed := to_float64_seconds(current_time_monotonic() - start);
should := elapsed >= 2 && elapsed < 2 + pause_seconds; should := elapsed >= 2 && elapsed < 2 + pause_seconds;
@ -237,7 +182,6 @@ main :: () {
last, width, height := record_last_frame(*recorder); last, width, height := record_last_frame(*recorder);
if mover DestroyWindow(mover); if mover DestroyWindow(mover);
if flash DestroyWindow(flash);
if cursor_test SetCursorPos(saved_cursor.x, saved_cursor.y); if cursor_test SetCursorPos(saved_cursor.x, saved_cursor.y);
stopped, stop_error := record_stop(*recorder); stopped, stop_error := record_stop(*recorder);
s := recorder.stats; s := recorder.stats;
@ -257,29 +201,6 @@ main :: () {
print(" each: copy % (per image), read back % , encode % (per frame)\n", print(" each: copy % (per image), read back % , encode % (per frame)\n",
ms_each(s.copy_ticks, s.captured), ms_each(s.readback_ticks, s.emitted), ms_each(s.encode_ticks, s.written)); ms_each(s.copy_ticks, s.captured), ms_each(s.readback_ticks, s.emitted), ms_each(s.encode_ticks, s.written));
print(" file: % KB, % MB a minute\n", file_size / 1024, FF(cast(float64) file_size / (1024 * 1024) / s.seconds * 60, 1)); print(" file: % KB, % MB a minute\n", file_size / 1024, FF(cast(float64) file_size / (1024 * 1024) / s.seconds * 60, 1));
if options.system_audio || options.microphone || options.audio_track {
print(" sound: %; % AAC frames%\n", ifx s.audio then s.audio else "none opened", s.audio_frames, ifx s.audio_errors then tprint(" (%)", s.audio_errors) else "");
sound := sound_report(options.path);
if sound.found print(" in file: % s of sound, % dB RMS, % onsets\n", FF(sound.seconds, 2), FF(sound.rms_db, 1), sound.onsets.count);
else print(" in file: no sound stream %\n", sound.error);
if sync_test && sound.found {
flashes := flash_times(options.path);
print(" sync: % flashes, % beeps\n", flashes.count, sound.onsets.count);
offsets: [..] float64;
offsets.allocator = temp;
for flash_time: flashes {
best: float64 = 1000000000.0;
for sound.onsets if abs(it - flash_time) < abs(best) best = it - flash_time;
if abs(best) < 0.5 array_add(*offsets, best);
}
if offsets.count {
text: [..] string; text.allocator = temp;
for offsets array_add(*text, tprint("%", FF(it * 1000, 0)));
print(" offsets: beep after flash, ms: %\n", join(..text, " ",, temp));
print(" drift: % ms from the first to the last\n", FF((offsets[offsets.count - 1] - offsets[0]) * 1000, 0));
}
}
}
if exclude_test { if exclude_test {
excluded_rect := window_rect_in(excluded, region); excluded_rect := window_rect_in(excluded, region);
@ -477,134 +398,6 @@ FF :: (value: float64, decimals: s64) -> FormatFloat #expand {
SetProcessDpiAwarenessContext :: (value: HANDLE) -> BOOL #foreign user32_dll; SetProcessDpiAwarenessContext :: (value: HANDLE) -> BOOL #foreign user32_dll;
SetWindowDisplayAffinity :: (window: HWND, affinity: u32) -> BOOL #foreign user32_dll; SetWindowDisplayAffinity :: (window: HWND, affinity: u32) -> BOOL #foreign user32_dll;
SetCursorPos :: (x: s32, y: s32) -> BOOL #foreign user32_dll; SetCursorPos :: (x: s32, y: s32) -> BOOL #foreign user32_dll;
// ---------------------------------------------------------------- sound and sync
// A beep of 1 kHz for 150 ms, as a WAV in memory, for PlaySound.
beep_wav :: () -> [] u8 {
RATE :: 48000;
COUNT :: RATE * 150 / 1000;
data := NewArray(44 + COUNT * 2, u8,, context.default_allocator);
put32 :: (at: s64, v: u32) #expand { for 0..3 `data[at + it] = cast,trunc(u8) (v >> (8 * it)); }
put16 :: (at: s64, v: u16) #expand { `data[at] = cast,trunc(u8) v; `data[at + 1] = cast,trunc(u8) (v >> 8); }
memcpy(data.data, "RIFF".data, 4); put32(4, cast(u32) (36 + COUNT * 2)); memcpy(data.data + 8, "WAVEfmt ".data, 8);
put32(16, 16); put16(20, 1); put16(22, 1); put32(24, RATE); put32(28, RATE * 2); put16(32, 2); put16(34, 16);
memcpy(data.data + 36, "data".data, 4); put32(40, COUNT * 2);
for i: 0..COUNT - 1 {
v := cast(s16) (sin(cast(float) i * 2 * PI * 1000 / RATE) * 20000);
put16(44 + i * 2, cast,no_check(u16) v);
}
return data;
}
// What the file's sound holds: its length, its level, and when it goes from quiet to loud (onsets).
Sound_Report :: struct { found: bool; seconds: float64; rms_db: float64; onsets: [..] float64; error: string; }
sound_report :: (path: string) -> Sound_Report {
report: Sound_Report;
report.onsets.allocator = temp;
format: *AVFormatContext;
if avformat_open_input(*format, temp_c_string(path), null, null) < 0 { report.error = "cannot open"; return report; }
defer avformat_close_input(*format);
avformat_find_stream_info(format, null);
index := -1;
for 0..format.nb_streams - 1 if format.streams[it].codecpar.codec_type == .AUDIO { index = it; break; }
if index < 0 return report;
report.found = true;
stream := format.streams[index];
codec := avcodec_find_decoder(stream.codecpar.codec_id);
c := avcodec_alloc_context3(codec);
defer avcodec_free_context(*c);
avcodec_parameters_to_context(c, stream.codecpar);
if avcodec_open2(c, codec, null) < 0 { report.error = "no decoder"; return report; }
packet := av_packet_alloc(); defer av_packet_free(*packet);
frame := av_frame_alloc(); defer av_frame_free(*frame);
WINDOW :: 480; // 10 ms at 48 kHz
sum_all: float64; count_all: s64;
window_sum: float64; window_count: s64; window_start: float64 = -1;
quiet_for: s64 = 100; // windows in a row below the threshold
first_time: float64 = -1; last_time: float64;
handle :: () #expand {
while avcodec_receive_frame(`c, `frame) >= 0 {
t := cast(float64) `frame.pts * `stream.time_base.num / `stream.time_base.den;
if `first_time < 0 `first_time = t;
left := cast(*float) `frame.data[0];
for i: 0..`frame.nb_samples - 1 {
v := cast(float64) left[i];
`sum_all += v * v; `count_all += 1;
if `window_start < 0 `window_start = t + cast(float64) i / 48000;
`window_sum += v * v; `window_count += 1;
if `window_count == WINDOW {
rms := sqrt(`window_sum / WINDOW);
if rms > 0.05 {
if `quiet_for >= 20 array_add(*`report.onsets, `window_start); // after 200 ms of quiet
`quiet_for = 0;
} else {
`quiet_for += 1;
}
`window_sum, `window_count, `window_start = 0, 0, -1;
}
}
`last_time = t + cast(float64) `frame.nb_samples / 48000;
}
}
while av_read_frame(format, packet) >= 0 {
if packet.stream_index == index { avcodec_send_packet(c, packet); handle(); }
av_packet_unref(packet);
}
avcodec_send_packet(c, null);
handle();
report.seconds = last_time - max(first_time, 0);
report.rms_db = ifx count_all && sum_all > 0 then 10 * log(sum_all / count_all) / log(10.0) else -200;
return report;
}
// When the video's middle turns white: the sync test's flashes, as times in seconds.
flash_times :: (path: string) -> [..] float64 {
times: [..] float64;
times.allocator = temp;
format: *AVFormatContext;
if avformat_open_input(*format, temp_c_string(path), null, null) < 0 return times;
defer avformat_close_input(*format);
avformat_find_stream_info(format, null);
index := -1;
for 0..format.nb_streams - 1 if format.streams[it].codecpar.codec_type == .VIDEO { index = it; break; }
if index < 0 return times;
stream := format.streams[index];
codec := avcodec_find_decoder(stream.codecpar.codec_id);
c := avcodec_alloc_context3(codec);
defer avcodec_free_context(*c);
avcodec_parameters_to_context(c, stream.codecpar);
if avcodec_open2(c, codec, null) < 0 return times;
packet := av_packet_alloc(); defer av_packet_free(*packet);
frame := av_frame_alloc(); defer av_frame_free(*frame);
was_white := false;
handle :: () #expand {
while avcodec_receive_frame(`c, `frame) >= 0 {
// The luma of the middle 32 x 32 (the first plane in every YUV format the encoders give).
w, h := `frame.width, `frame.height;
sum: s64;
for y: h / 2 - 16..h / 2 + 15 for x: w / 2 - 16..w / 2 + 15 sum += `frame.data[0][y * `frame.linesize[0] + x];
white := sum / 1024 > 200;
if white && !`was_white array_add(*`times, cast(float64) `frame.best_effort_timestamp * `stream.time_base.num / `stream.time_base.den);
`was_white = white;
}
}
while av_read_frame(format, packet) >= 0 {
if packet.stream_index == index { avcodec_send_packet(c, packet); handle(); }
av_packet_unref(packet);
}
avcodec_send_packet(c, null);
handle();
return times;
}
PlaySoundW :: (sound: *void, module: *void, flags: u32) -> s32 #foreign winmm_dll;
winmm_dll :: #library,system "winmm";
SND_ASYNC :: 0x1;
SND_MEMORY :: 0x4;
user32_dll :: #library,system "user32"; user32_dll :: #library,system "user32";
#import "Basic"; #import "Basic";
@ -617,4 +410,3 @@ user32_dll :: #library,system "user32";
#import "Screen_Record"; #import "Screen_Record";
#import "Image_Decode"; #import "Image_Decode";
#import "stb_image_write"; #import "stb_image_write";
#import "ffmpeg";

View File

@ -68,7 +68,7 @@
<header> <header>
<h1>Screen video capture in mcap</h1> <h1>Screen video capture in mcap</h1>
<p>Planning document · 2026-10-10 · for mcap after 0.1.1 (still captures with PrintScreen) · <strong>all five phases built, mcap 0.1.8</strong></p> <p>Planning document · 2026-10-10 · for mcap after 0.1.1 (still captures with PrintScreen) · <strong>phases 1 to 4 built, mcap 0.1.6</strong></p>
</header> </header>
<div class="summary"> <div class="summary">
@ -83,7 +83,6 @@
<li><span class="tag decided">Done</span> <strong>Phase 1</strong> (2026-10-10, mcap 0.1.3): <code>modules/Screen_Record</code> and <code>record-probe</code>. 60 s of 4K at 30 fps with NVENC and no dropped frames. The two-GPU failure couldn't be reproduced, because this laptop's screens are currently driven by the NVIDIA GPU (see <a href="#phases">phase 1's result</a>).</li> <li><span class="tag decided">Done</span> <strong>Phase 1</strong> (2026-10-10, mcap 0.1.3): <code>modules/Screen_Record</code> and <code>record-probe</code>. 60 s of 4K at 30 fps with NVENC and no dropped frames. The two-GPU failure couldn't be reproduced, because this laptop's screens are currently driven by the NVIDIA GPU (see <a href="#phases">phase 1's result</a>).</li>
<li><span class="tag decided">Done</span> <strong>Phase 2</strong> (2026-10-10, mcap 0.1.4): the Windows.Graphics.Capture fallback. Automatic mode falls back to it on a simulated <code>DXGI_ERROR_UNSUPPORTED</code>. It records 4K at 60 fps with nothing dropped, draws the cursor exactly as Desktop Duplication's code does, and the yellow border can be turned off on Windows 11.</li> <li><span class="tag decided">Done</span> <strong>Phase 2</strong> (2026-10-10, mcap 0.1.4): the Windows.Graphics.Capture fallback. Automatic mode falls back to it on a simulated <code>DXGI_ERROR_UNSUPPORTED</code>. It records 4K at 60 fps with nothing dropped, draws the cursor exactly as Desktop Duplication's code does, and the yellow border can be turned off on Windows 11.</li>
<li><span class="tag decided">Done</span> <strong>Phase 3</strong> (2026-10-10, mcap 0.1.5): recording in the app. Ctrl+PrintScreen opens the overlay's video mode; Ctrl+PrintScreen again stops; the recording becomes a video <code>.mcap</code> in Screen Captures. In the app, Desktop Duplication first failed with <code>DXGI_ERROR_UNSUPPORTED</code>: only the main thread was DPI aware. The capture thread now makes itself aware.</li> <li><span class="tag decided">Done</span> <strong>Phase 3</strong> (2026-10-10, mcap 0.1.5): recording in the app. Ctrl+PrintScreen opens the overlay's video mode; Ctrl+PrintScreen again stops; the recording becomes a video <code>.mcap</code> in Screen Captures. In the app, Desktop Duplication first failed with <code>DXGI_ERROR_UNSUPPORTED</code>: only the main thread was DPI aware. The capture thread now makes itself aware.</li>
<li><span class="tag decided">Done</span> <strong>Phase 5</strong> (2026-10-10, mcap 0.1.8): sound. What the computer plays, and the microphone if asked for, are mixed in sync with the video into an AAC stream. A beep matched to a flash every 2 s stays within about 20 ms, without drifting.</li>
<li><span class="tag decided">Done</span> <strong>Phase 4</strong> (2026-10-10, mcap 0.1.6): the border and toolbar, both kept out of the recording; pause, with no gap in the video; discard, confirmed in the toolbar; recovery when the screen is lost; leftover recordings offered at startup; the <code>[video]</code> settings.</li> <li><span class="tag decided">Done</span> <strong>Phase 4</strong> (2026-10-10, mcap 0.1.6): the border and toolbar, both kept out of the recording; pause, with no gap in the video; discard, confirmed in the toolbar; recovery when the screen is lost; leftover recordings offered at startup; the <code>[video]</code> settings.</li>
<li><span class="tag later">Later</span> Audio is phase 5. Recording one window, and regions that span two screens, come after.</li> <li><span class="tag later">Later</span> Audio is phase 5. Recording one window, and regions that span two screens, come after.</li>
</ul> </ul>
@ -435,12 +434,6 @@ Captured_Frame :: struct {
<li>Leftover recordings offered at startup, in a popup in the main window.</li> <li>Leftover recordings offered at startup, in a popup in the main window.</li>
<li>The Win32 GDI bindings moved to <code>src/base/platform/win32_gdi_bindings.jai</code>, shared by the overlay and the toolbar.</li> <li>The Win32 GDI bindings moved to <code>src/base/platform/win32_gdi_bindings.jai</code>, shared by the overlay and the toolbar.</li>
</ul></li> </ul></li>
<li><strong>Found by hand: a killed recording's timestamps were wrong</strong> (mcap 0.1.7). There were two causes:
<ul>
<li><strong>No length or index.</strong> A fragmented MP4's header gives its length as 0, and one cut short never gets the index a clean stop writes at its end, so players showed its time and seeked badly. Every recording, cut short or not, is now rewritten as a regular MP4 on its way into the library (<code>record_remux</code>): its packets copied as they are, with one header first holding its length and an index of every frame. Nothing is re-encoded.</li>
<li><strong>Keyframes counted in frames.</strong> They came every 60 frames, and with frames written only on change, 60 frames can span minutes. So a fragment, and what a crash loses, could span minutes too, not the 2 s section 5 promised. A keyframe is now forced every 2 s of time.</li>
</ul>
Checked: your recording's MP4, cut mid-fragment, rewrites to a regular MP4 of the 76 whole frames (5.2 s) with positive frame durations. A 12 s mostly still recording has fragments at 0, 2.0, 4.1, 6.1, 8.1, 10.2 and 12.1 s. The app killed about 10 s into a recording leaves 8.1 s. A test cuts a recording at its last fragment, as a crash would, and checks it goes in as a regular MP4 of the time before the cut.</li>
<li><strong>Kept out of the recording:</strong> a whole-screen recording with the app (<code>-record 6</code>), where the toolbar sits inside the recorded screen. The frame stored in its <code>.mcap</code> has no toolbar. With <code>-record-ui-visible</code> and a region, a screenshot shows both drawn as designed: the frame just outside the region, and the toolbar below it at its right.</li> <li><strong>Kept out of the recording:</strong> a whole-screen recording with the app (<code>-record 6</code>), where the toolbar sits inside the recorded screen. The frame stored in its <code>.mcap</code> has no toolbar. With <code>-record-ui-visible</code> and a region, a screenshot shows both drawn as designed: the frame just outside the region, and the toolbar below it at its right.</li>
<li><strong>Pause:</strong> 16 s with a 10 s pause from 2 s in gives a 6.2 s video at a steady 30 fps, with Desktop Duplication and with Windows.Graphics.Capture alike. The pause is cut out, with no gap.</li> <li><strong>Pause:</strong> 16 s with a 10 s pause from 2 s in gives a 6.2 s video at a steady 30 fps, with Desktop Duplication and with Windows.Graphics.Capture alike. The pause is cut out, with no gap.</li>
<li><strong>Losing the screen</strong> (<code>-lose</code>, which loses Desktop Duplication every 3 s through the real recovery path): 3 losses in 10 s, 304 frames, 30 fps throughout. A real UAC prompt or resolution change is still to try by hand.</li> <li><strong>Losing the screen</strong> (<code>-lose</code>, which loses Desktop Duplication every 3 s through the real recovery path): 3 losses in 10 s, 304 frames, 30 fps throughout. A real UAC prompt or resolution change is still to try by hand.</li>
@ -467,48 +460,9 @@ Captured_Frame :: struct {
</div> </div>
<div class="phase"> <div class="phase">
<h3>Phase 5: Audio <span class="tag decided">Done</span></h3> <h3>Phase 5: Audio</h3>
<p>Section 8: <code>swresample</code> bindings, loopback and microphone, mixing, AAC.</p> <p>Section 8: <code>swresample</code> bindings, loopback and microphone, mixing, AAC.</p>
<p><strong>Done when:</strong> a 10-minute recording with a video playing and someone talking stays in sync to within 50 ms at the end, and silence doesn't shift it.</p> <p><strong>Done when:</strong> a 10-minute recording with a video playing and someone talking stays in sync to within 50 ms at the end, and silence doesn't shift it.</p>
<div class="note"><strong>Result (2026-10-10).</strong>
<ul>
<li><strong>Built:</strong>
<ul>
<li><code>modules/Screen_Record/audio_source.jai</code>: WASAPI for both sources. What the computer plays comes in loopback from the default output device; the microphone is the default input device.</li>
<li>An audio thread in <code>module.jai</code> that mixes them.</li>
<li>An AAC stream in <code>encoder.jai</code>, with the two threads' packets meeting in the file under a lock.</li>
<li><code>system_audio</code> and <code>microphone</code> in <code>[video]</code>, defaulting to true and false.</li>
</ul></li>
<li><strong>Sync, by measurement:</strong> <code>record-probe -sync-test</code> flashes a white square and plays a 1 kHz beep together every 2 s, then finds both in the file.
<ul>
<li>20 s with system sound: 10 flashes, 10 beeps, each beep within 0 to 21 ms of its flash, with no drift (−10 ms first to last, within the spread).</li>
<li>Both sources mixed: within 5 to 18 ms.</li>
<li>With a 6 s pause: sound and video lose the same 6 s (14.1 s of sound, 14.2 s of video), and the beeps after the pause line up as before.</li>
<li>3 minutes with system sound: 92 flashes, 90 beeps found. 85 beeps were within 3 to 24 ms of their flash, and from the first to the last the offset moved −14 ms: no drift. Five were 53 to 147 ms off, in two clusters about 40 s and 60 s in, near where the two beeps went missing. That points to Windows playing the test beep late or not at all then, but it isn't proven; a 10-minute run with someone talking, as the done-when asks, is still to do by hand.</li>
</ul>
The offsets include Windows' own delay in playing the beep, so they are an upper bound on the true offset.</li>
<li><strong>In the app:</strong> <code>-record 6</code> recorded the monitor's speakers. The <code>.mcap</code>'s MP4 has its AAC stream, and its <code>capture.json</code> has <code>Audio: true</code> and the device's name. Your settings file gained <code>system_audio</code> and <code>microphone</code>.</li>
<li><strong>Tests:</strong> 4 more checks, 30 in the recording group. An MP4 made with sound by the module's encoder keeps its AAC stream through the remux into the library, and <code>capture.json</code> names its source. Every suite passes.</li>
</ul>
<p><strong>Changes from the plan:</strong></p>
<ul>
<li><strong>No <code>swresample</code>.</strong> WASAPI converts any device to 48 kHz stereo float itself (<code>AUTOCONVERTPCM</code>), so no bindings were generated. A device that won't convert is taken as it is if it gives 48 kHz float, its channels made stereo here.</li>
<li><strong>How the timeline holds together:</strong>
<ul>
<li>Each packet is placed by the time Windows says it was heard, on the video's own timeline: the same time 0, less the same pauses. <code>record_pause</code> now keeps one account of time paused for both threads.</li>
<li>A packet within 30 ms of where its source left off follows on exactly, so timing jitter doesn't leave clicks.</li>
<li>A source that sends nothing, as loopback does while nothing plays, is silence, and the mix moves on without it 200 ms late.</li>
</ul></li>
<li><strong>The microphone in the toolbar</strong> (added in mcap 0.1.9, on request).
<ul>
<li><strong>Mic: on / off</strong> turns the microphone on and off mid-recording (<code>record_set_microphone</code>), and its state is kept as <code>microphone</code> in <code>[video]</code> for the next recording.</li>
<li>The microphone is opened only while on, so Windows' microphone-in-use indicator shows only then.</li>
<li>An MP4 can't gain a sound stream once its header is written, so the app's recordings always have one (<code>audio_track</code>), silent while nothing is on.</li>
<li>Checked with <code>record-probe -mic-toggle</code>, which turns the microphone on at 3 s and off at 7 s. With system sound, and with nothing else on, the sound track runs the whole recording without a break.</li>
</ul>
In mcap 0.1.10, <strong>System sound: on / off</strong> joined it and works the same way, saved as <code>system_audio</code>. With <code>record-probe -system-toggle -sync-test</code> (system sound on at 3 s, off at 7 s), only the beeps at 3 and 5 s are in the file, 8 to 12 ms from their flashes, and the track runs the whole recording.</li>
<li><strong>A lost device</strong> (a headset unplugged, the default changed) is opened again from the new default once a second.</li>
</ul></div>
</div> </div>
<h2 id="risks">10. Risks</h2> <h2 id="risks">10. Risks</h2>