Musa Mahmood
72f2754226
- modules/Screen_Record/audio_source.jai: WASAPI, in loopback from the default output device (what the computer plays) and from the default input device (the microphone), asked for as 48 kHz stereo float, which Windows converts any device to; a device that will not convert is taken as it is when it gives 48 kHz float. Packets carry the time they were heard. A device lost (unplugged, default changed) is opened again from the new default once a second. - An audio thread mixes them onto the video's own timeline: the same time 0, less the same pauses (record_pause now keeps one account of time paused for both threads). A packet within 30 ms of where its source left off follows on exactly; a source that sends nothing is silence, passed 200 ms late. Each 1024 samples every source has covered become an AAC frame. - encoder.jai: an AAC stream (FFmpeg's aac, 160 kb/s) beside the video, written from the audio thread; both threads' packets meet in the file under a lock. The remux keeps it. - [video] system_audio (true) and microphone (false). capture.json: Audio, AudioSources. A toast when a source asked for could not be opened. - record-probe: -audio system|mic|both, the sound read back (length, level), and -sync-test, which flashes a white square and beeps together every 2 s, then finds both in the file. - Measured: 20 s, beeps 0-21 ms from their flashes, no drift; both sources mixed, 5-18 ms; a 6 s pause cuts the same 6 s from sound and video; 3 min, 85 of 90 beeps within 3-24 ms and -14 ms from first to last, with 5 outliers (53-147 ms) where two beeps went missing. Tests: an MP4 with sound keeps its AAC stream through the remux into the library (30 checks).
729 lines
30 KiB
Plaintext
729 lines
30 KiB
Plaintext
// Screen_Record: a region of one screen recorded to H.264 in an MP4, for the mcap app's screen
|
|
// recording and tools/record_probe.jai (video-capture-plan.html). Windows only.
|
|
//
|
|
// Two threads. The capture thread takes the screen with DXGI Desktop Duplication (dxgi_source.jai),
|
|
// or with Windows.Graphics.Capture (wgc_source.jai) where duplication cannot run, each on a D3D11
|
|
// device of its own; copies the region on the GPU; reads it back through a ring of staging textures,
|
|
// so Map never waits on the GPU (capture_gpu.jai); and, for duplication, draws the cursor in
|
|
// (cursor.jai). The encoder thread hands the frames to FFmpeg (encoder.jai). Between them is a pool of
|
|
// RECORD_BUFFERS frames; when the encoder falls RECORD_QUEUE frames behind, the oldest waiting is
|
|
// dropped and counted.
|
|
//
|
|
// A frame is taken at most fps times a second, and only when the screen or the cursor changed, with
|
|
// one a second while nothing does, so players can seek. Each carries the time it was taken, less the
|
|
// time spent paused (record_pause), so a pause leaves no gap in the video.
|
|
//
|
|
// Sound (system_audio, microphone) has a thread of its own (audio_source.jai's WASAPI): each packet is
|
|
// put on the same timeline as the frames, by when it was heard, less the same pauses, into one mix,
|
|
// and an AAC frame of it is encoded once every source has had its say up to there. A source that sends
|
|
// nothing (the computer playing nothing) is silence.
|
|
//
|
|
// recorder: Recorder;
|
|
// ok, error, fall_back := record_start(*recorder, .{path = "out.mp4", screen = 0});
|
|
// ...
|
|
// ok, error = record_stop(*recorder);
|
|
//
|
|
// With capture .AUTO (the default), Desktop Duplication is tried first and Windows.Graphics.Capture
|
|
// takes over when it cannot run: on a laptop with two GPUs, on a rotated screen, or with too many
|
|
// duplications of the screen running. fall_back is only returned for .DXGI, which does not fall back.
|
|
|
|
RECORD_MIN_WIDTH :: 240; // a smaller region is refused
|
|
RECORD_MIN_HEIGHT :: 180;
|
|
RECORD_QUEUE :: 8; // frames waiting for the encoder at most
|
|
RECORD_BUFFERS :: RECORD_QUEUE + 2; // and the one being filled, and the one being encoded
|
|
|
|
// Whether a region this size may be recorded: at least RECORD_MIN_WIDTH x RECORD_MIN_HEIGHT. It is
|
|
// rounded down to even sizes after, which keeps it at least that, as both are even.
|
|
record_size_allowed :: (width: s64, height: s64) -> bool {
|
|
return width >= RECORD_MIN_WIDTH && height >= RECORD_MIN_HEIGHT;
|
|
}
|
|
|
|
Record_Screen :: struct {
|
|
adapter_index, output_index: u32;
|
|
adapter: string; // "NVIDIA GeForce RTX 5080 Laptop GPU"
|
|
name: string; // "\\.\DISPLAY1"
|
|
desktop: RECT; // on the virtual desktop
|
|
rotation: DXGI_MODE_ROTATION;
|
|
monitor: HMONITOR;
|
|
}
|
|
|
|
Record_Capture :: enum u8 {
|
|
AUTO; // Desktop Duplication, else Windows.Graphics.Capture
|
|
DXGI; // Desktop Duplication only
|
|
WGC; // Windows.Graphics.Capture only
|
|
}
|
|
|
|
// For record-probe: paths a machine may not take by itself.
|
|
Record_Testing :: enum_flags u8 {
|
|
DXGI_UNSUPPORTED; // Desktop Duplication fails as on a laptop with two GPUs (DXGI_ERROR_UNSUPPORTED)
|
|
WGC_POLL; // Windows.Graphics.Capture polled, as when it will not take the FrameArrived handler
|
|
KEEP_BORDER; // Windows.Graphics.Capture's yellow border left on
|
|
LOSE_DUPLICATION; // Desktop Duplication lost every 3 s, as a mode change or the secure desktop loses it
|
|
}
|
|
|
|
Record_Options :: struct {
|
|
path: string; // the .mp4 written
|
|
screen: s32 = -1; // into record_list_screens(); -1: the one holding region
|
|
region: RECT; // on the virtual desktop; empty: the whole screen
|
|
fps: s32 = 30;
|
|
cursor := true;
|
|
encoder: string; // "" tries RECORD_ENCODERS in turn
|
|
capture: Record_Capture;
|
|
legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe
|
|
testing: Record_Testing;
|
|
system_audio: bool; // what the computer plays (the default output device)
|
|
microphone: bool; // the default input device
|
|
}
|
|
|
|
Record_Stats :: struct {
|
|
source: string; // "DXGI DuplicateOutput1 on <adapter>", "Windows.Graphics.Capture on <adapter>, border off"
|
|
fell_back_from: string; // why Desktop Duplication could not run, when it fell back
|
|
encoder: string;
|
|
width, height: s32;
|
|
captured: s64; // new images of the screen
|
|
pointer_moves: s64; // cursor changes with no new image (Desktop Duplication)
|
|
emitted: s64; // frames sent to be encoded
|
|
repeats: s64; // of them, the once-a-second repeat of an unchanged screen
|
|
written: s64; // encoded
|
|
dropped: s64; // the encoder was too far behind
|
|
lost: s64; // times the duplication was lost and made again
|
|
paused_seconds: float64; // left out of the video
|
|
audio: string; // the sound recorded: "Speakers (Realtek(R) Audio); Microphone (USB Audio)", or ""
|
|
audio_frames: s64; // AAC frames encoded
|
|
audio_errors: string; // sources that could not be opened, or were lost
|
|
copy_ticks, readback_ticks, encode_ticks: s64; // totals, in QueryPerformanceCounter ticks
|
|
bytes: s64; // encoded
|
|
seconds: float64; // from start to stop
|
|
}
|
|
|
|
Recorder :: struct {
|
|
options: Record_Options;
|
|
screen: Record_Screen;
|
|
local: RECT; // the region, in the screen's pixels
|
|
dxgi: Dxgi_Source; // the capture thread's: one of these is open
|
|
wgc: Wgc_Source;
|
|
using_wgc: bool;
|
|
encoder: Encoder;
|
|
width, height: s32;
|
|
frequency: s64; // QueryPerformanceCounter's
|
|
start_ticks: s64;
|
|
|
|
capture_thread, encoder_thread: Thread;
|
|
capture_started, encoder_started, audio_started: bool;
|
|
audio_thread: Thread;
|
|
audio_opened: Semaphore; // the audio thread has opened its sources
|
|
audio_go: Semaphore; // the encoder is open: the audio thread may write
|
|
audio_sources: [2] Audio_Source;
|
|
audio_open: bool; // any source opened: the MP4 has sound
|
|
zero_ticks: s64; // the recording's time 0, for frames and sound alike
|
|
paused_ticks: s64; // time paused so far, for the capture and audio threads; set by record_pause
|
|
pause_began: s64;
|
|
running: bool;
|
|
stopping: bool; // set by record_stop; the capture thread ends, then the encoder
|
|
paused: bool; // set by record_pause; the capture thread takes nothing while it is
|
|
capture_ended: bool;
|
|
|
|
opened: Semaphore; // the capture thread has opened its source, or failed to
|
|
open_ok, open_fall_back: bool;
|
|
open_error: string;
|
|
|
|
mutex: Mutex;
|
|
ready: Semaphore; // signalled for each frame queued, and once when capture ends
|
|
frames: [RECORD_BUFFERS] Record_Frame;
|
|
free: [RECORD_BUFFERS] s32;
|
|
free_count: s32;
|
|
queue: [RECORD_BUFFERS] s32; // oldest first
|
|
queue_count: s32;
|
|
last_frame: [] u8; // the last frame queued, BGRA, for record_last_frame
|
|
stats: Record_Stats;
|
|
error: string; // why it stopped by itself; heap
|
|
}
|
|
|
|
Record_Frame :: struct {
|
|
pixels: [] u8; // BGRA, rows packed
|
|
ticks: s64;
|
|
}
|
|
|
|
// Opens the screen, the encoder and the file, and starts recording.
|
|
record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: string, fall_back: bool {
|
|
r.* = .{};
|
|
r.options = options;
|
|
r.options.path = copy_string(options.path);
|
|
r.options.encoder = copy_string(options.encoder);
|
|
if r.options.fps <= 0 r.options.fps = 30;
|
|
|
|
screens := record_list_screens();
|
|
if !screens.count return false, "no screen to record", false;
|
|
screen_index := options.screen;
|
|
region := options.region;
|
|
has_region := region.right > region.left && region.bottom > region.top;
|
|
if screen_index < 0 {
|
|
if !has_region screen_index = 0;
|
|
else for screens if rect_inside(region, it.desktop) { screen_index = xx it_index; break; }
|
|
if screen_index < 0 return false, "the region is not on one screen", false;
|
|
}
|
|
if screen_index >= screens.count return false, tprint("there is no screen %", screen_index), false;
|
|
r.screen = screens[screen_index];
|
|
r.screen.adapter = copy_string(r.screen.adapter);
|
|
r.screen.name = copy_string(r.screen.name);
|
|
if !has_region region = r.screen.desktop;
|
|
if !rect_inside(region, r.screen.desktop) return false, "the region is not on that screen", false;
|
|
|
|
// The minimum first, then even both ways, in the screen's pixels: H.264's 4:2:0 colour needs it.
|
|
// RECORD_MIN_WIDTH and _HEIGHT are even, so what passes stays at least that size.
|
|
#assert RECORD_MIN_WIDTH % 2 == 0 && RECORD_MIN_HEIGHT % 2 == 0;
|
|
if !record_size_allowed(region.right - region.left, region.bottom - region.top)
|
|
return false, tprint("% x % is smaller than % x %", region.right - region.left, region.bottom - region.top, RECORD_MIN_WIDTH, RECORD_MIN_HEIGHT), false;
|
|
d := r.screen.desktop;
|
|
r.local = .{region.left - d.left, region.top - d.top, region.right - d.left, region.bottom - d.top};
|
|
r.local.right = r.local.left + ((r.local.right - r.local.left) & cast(s32) ~1);
|
|
r.local.bottom = r.local.top + ((r.local.bottom - r.local.top) & cast(s32) ~1);
|
|
r.width = r.local.right - r.local.left;
|
|
r.height = r.local.bottom - r.local.top;
|
|
r.stats.width, r.stats.height = r.width, r.height;
|
|
|
|
init(*r.mutex, "record");
|
|
init(*r.ready);
|
|
init(*r.opened);
|
|
for * r.frames it.pixels = NewArray(cast(s64) r.width * r.height * 4, u8, initialized = false);
|
|
for 0..RECORD_BUFFERS - 1 r.free[it] = cast(s32) it;
|
|
r.free_count = RECORD_BUFFERS;
|
|
r.last_frame = NewArray(cast(s64) r.width * r.height * 4, u8);
|
|
frequency: LARGE_INTEGER;
|
|
QueryPerformanceFrequency(*frequency);
|
|
r.frequency = frequency.QuadPart;
|
|
r.start_ticks = ticks_now();
|
|
r.zero_ticks = r.start_ticks;
|
|
|
|
// The source is opened on the capture thread, which uses it: WinRT's objects belong to the
|
|
// apartment they are made in.
|
|
thread_init(*r.capture_thread, record_capture_proc, name = "record capture");
|
|
r.capture_started = true;
|
|
thread_start(*r.capture_thread, r);
|
|
wait_for(*r.opened);
|
|
if !r.open_ok {
|
|
error, fall_back := copy_string(r.open_error,, temp), r.open_fall_back;
|
|
record_free(r);
|
|
return false, error, fall_back;
|
|
}
|
|
|
|
// The sound's sources are opened on their own thread (COM, for WASAPI), before the encoder, which
|
|
// has a sound stream only when one opened.
|
|
if r.options.system_audio || r.options.microphone {
|
|
init(*r.audio_opened);
|
|
init(*r.audio_go);
|
|
thread_init(*r.audio_thread, record_audio_proc, name = "record audio");
|
|
r.audio_started = true;
|
|
thread_start(*r.audio_thread, r);
|
|
wait_for(*r.audio_opened);
|
|
}
|
|
|
|
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder, audio = r.audio_open);
|
|
if tried log("[record] Encoders that did not open: %", tried);
|
|
if !ok {
|
|
error = copy_string(error,, temp);
|
|
record_free(r);
|
|
return false, error, false;
|
|
}
|
|
r.stats.encoder = copy_string(r.encoder.name);
|
|
if r.audio_started signal(*r.audio_go);
|
|
|
|
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
|
|
r.encoder_started = true;
|
|
r.running = true;
|
|
thread_start(*r.encoder_thread, r);
|
|
return true, "", false;
|
|
}
|
|
|
|
// Stops, finishes the file and frees everything. ok is false when recording stopped by itself on an
|
|
// error, or the file could not be finished; what was recorded up to then is in the file all the same.
|
|
record_stop :: (r: *Recorder) -> ok: bool, error: string {
|
|
if !r.running return false, "not recording";
|
|
atomic_swap(*r.stopping, true);
|
|
thread_is_done(*r.capture_thread, -1);
|
|
thread_is_done(*r.encoder_thread, -1);
|
|
if r.audio_started thread_is_done(*r.audio_thread, -1);
|
|
r.running = false;
|
|
|
|
closed, close_error := encoder_close(*r.encoder);
|
|
r.stats.seconds = cast(float64) (ticks_now() - r.start_ticks) / r.frequency;
|
|
r.stats.bytes = r.encoder.bytes;
|
|
error := ifx r.error then copy_string(r.error,, temp) else "";
|
|
record_free(r);
|
|
|
|
if error return false, error;
|
|
if !closed return false, close_error;
|
|
return true, "";
|
|
}
|
|
|
|
// Pauses or resumes. While paused nothing is taken, and the time is left out of the video, sound and
|
|
// all: the time paused is kept here, once, for the capture and audio threads both.
|
|
record_pause :: (r: *Recorder, pause: bool) {
|
|
if !r.running || atomic_read(*r.paused) == pause return;
|
|
now := ticks_now();
|
|
if pause {
|
|
atomic_write(*r.pause_began, now);
|
|
atomic_swap(*r.paused, true);
|
|
} else {
|
|
atomic_add(*r.paused_ticks, now - atomic_read(*r.pause_began)); // before paused clears, so no thread sees it unpaused with the old total
|
|
atomic_swap(*r.paused, false);
|
|
}
|
|
}
|
|
|
|
// Ticks on the recording's timeline: time 0 at its start, less the time paused.
|
|
record_timeline :: (r: *Recorder, ticks: s64) -> s64 {
|
|
return ticks - r.zero_ticks - atomic_read(*r.paused_ticks);
|
|
}
|
|
|
|
record_paused :: (r: *Recorder) -> bool {
|
|
return r.running && atomic_read(*r.paused);
|
|
}
|
|
|
|
// As they are now; strings are the recorder's.
|
|
record_stats :: (r: *Recorder) -> Record_Stats {
|
|
if !r.running return r.stats;
|
|
lock(*r.mutex);
|
|
defer unlock(*r.mutex);
|
|
return r.stats;
|
|
}
|
|
|
|
// The last frame queued, BGRA (temp), cursor and all.
|
|
record_last_frame :: (r: *Recorder) -> pixels: [] u8, width: s32, height: s32 {
|
|
if r.running lock(*r.mutex);
|
|
pixels := array_copy(r.last_frame,, temp);
|
|
if r.running unlock(*r.mutex);
|
|
return pixels, r.width, r.height;
|
|
}
|
|
|
|
// Whether recording stopped by itself (the screen went away, the encoder failed); why, then.
|
|
record_failed :: (r: *Recorder) -> bool, string {
|
|
lock(*r.mutex);
|
|
defer unlock(*r.mutex);
|
|
return r.error.count > 0, r.error;
|
|
}
|
|
|
|
#scope_file
|
|
|
|
Staged :: struct { pending: bool; ticks: s64; pointer: POINT; pointer_visible: bool; }
|
|
|
|
// Stops the capture thread if it runs, and frees what record_start made. The stats' strings are kept.
|
|
record_free :: (r: *Recorder) {
|
|
atomic_swap(*r.stopping, true);
|
|
if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; }
|
|
if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; }
|
|
if r.audio_started {
|
|
signal(*r.audio_go); // in case it waits for an encoder that did not open
|
|
thread_is_done(*r.audio_thread, -1);
|
|
thread_deinit(*r.audio_thread);
|
|
destroy(*r.audio_opened);
|
|
destroy(*r.audio_go);
|
|
r.audio_started = false;
|
|
}
|
|
if r.encoder.format encoder_close(*r.encoder);
|
|
for * r.frames array_free(it.pixels);
|
|
array_free(r.last_frame);
|
|
r.last_frame = .[];
|
|
destroy(*r.opened);
|
|
destroy(*r.ready);
|
|
destroy(*r.mutex);
|
|
free(r.open_error);
|
|
free(r.error);
|
|
r.open_error, r.error = "", "";
|
|
}
|
|
|
|
// Desktop Duplication, unless asked for Windows.Graphics.Capture, which takes over when duplication
|
|
// cannot run here and the options allow.
|
|
record_open_source :: (r: *Recorder) -> bool {
|
|
if r.options.capture != .WGC {
|
|
ok, error, fall_back := dxgi_open(*r.dxgi, r.screen, r.local, r.options.legacy_duplicate);
|
|
if ok && r.options.testing & .DXGI_UNSUPPORTED {
|
|
ok, error, fall_back = false, "Desktop Duplication (testing): 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", true;
|
|
}
|
|
if ok {
|
|
r.stats.source = sprint("DXGI % on %", r.dxgi.method, r.screen.adapter);
|
|
return true;
|
|
}
|
|
dxgi_close(*r.dxgi);
|
|
if r.options.capture == .DXGI || !fall_back {
|
|
r.open_error, r.open_fall_back = copy_string(error), fall_back;
|
|
return false;
|
|
}
|
|
log("[record] Desktop Duplication cannot run here (%); Windows.Graphics.Capture instead.", error);
|
|
r.stats.fell_back_from = copy_string(error);
|
|
}
|
|
ok, error := wgc_open(*r.wgc, r.screen, r.local, r.options.cursor, r.options.testing);
|
|
if !ok {
|
|
wgc_close(*r.wgc);
|
|
r.open_error = copy_string(error);
|
|
return false;
|
|
}
|
|
r.using_wgc = true;
|
|
r.stats.source = sprint("Windows.Graphics.Capture on %, %%", r.screen.adapter, r.wgc.border, ifx r.wgc.polling then ", polled" else "");
|
|
return true;
|
|
}
|
|
|
|
record_capture_proc :: (thread: *Thread) -> s64 {
|
|
r := cast(*Recorder) thread.data;
|
|
// Aware of each monitor's DPI, whatever the process is: Desktop Duplication refuses a thread that is
|
|
// not (DXGI_ERROR_UNSUPPORTED), and the mcap app's ImGui makes only its main thread aware.
|
|
gpu_thread_dpi_aware();
|
|
wgc_thread_start();
|
|
defer wgc_thread_end();
|
|
r.open_ok = record_open_source(r);
|
|
signal(*r.opened);
|
|
if !r.open_ok return 0;
|
|
gpu: *Capture_Gpu = ifx r.using_wgc then *r.wgc.gpu else *r.dxgi.gpu;
|
|
defer if r.using_wgc wgc_close(*r.wgc); else dxgi_close(*r.dxgi);
|
|
|
|
// A change is sent as soon as it comes once its turn is due: turns are a frame's time apart on a
|
|
// schedule that moves on a frame at a time, so the rate never passes fps, and a tenth of a frame
|
|
// early counts as on time, so a 60 Hz screen's frames, which jitter, all make a 60 fps cap.
|
|
period := r.frequency / r.options.fps;
|
|
early := period / 10;
|
|
turn := ticks_now();
|
|
last_emit: s64 = -r.frequency;
|
|
have_image, changed := false, false;
|
|
staged: [RECORD_STAGING] Staged;
|
|
emits: s64;
|
|
was_paused := false;
|
|
last_lost := ticks_now(); // for Record_Testing.LOSE_DUPLICATION
|
|
|
|
while !atomic_read(*r.stopping) {
|
|
now := ticks_now();
|
|
due := ifx changed then turn - early else last_emit + r.frequency; // the change, or the repeat
|
|
wait := cast(u32) clamp((due - now) * 1000 / r.frequency, 0, 100);
|
|
next: Capture_Next;
|
|
error: string;
|
|
if r.using_wgc next, error = wgc_next(*r.wgc, wait);
|
|
else next, error = dxgi_next(*r.dxgi, wait);
|
|
if r.options.testing & .LOSE_DUPLICATION && !r.using_wgc && ticks_now() - last_lost >= 3 * r.frequency {
|
|
next, last_lost = Capture_Next.LOST, ticks_now();
|
|
}
|
|
if next == {
|
|
case .FAILED;
|
|
record_fail(r, error);
|
|
break;
|
|
case .LOST;
|
|
// A mode change, the secure desktop, a game going fullscreen: duplicated again, and on.
|
|
lock(*r.mutex); r.stats.lost += 1; unlock(*r.mutex);
|
|
while !atomic_read(*r.stopping) {
|
|
hr := dxgi_duplicate(*r.dxgi);
|
|
if SUCCEEDED(hr) break;
|
|
if !hresult_is(hr, DXGI_ERROR_ACCESS_LOST) && !hresult_is(hr, 0x80070005) && !hresult_is(hr, DXGI_ERROR_NOT_CURRENTLY_AVAILABLE) {
|
|
record_fail(r, hresult_text("the screen could not be duplicated again", hr));
|
|
break;
|
|
}
|
|
sleep_milliseconds(250);
|
|
}
|
|
if atomic_read(*r.stopping) || r.error break;
|
|
continue;
|
|
case .IMAGE; #through;
|
|
case .IMAGE_AND_POINTER;
|
|
have_image, changed = true, true;
|
|
lock(*r.mutex);
|
|
r.stats.captured += 1;
|
|
r.stats.copy_ticks = gpu.copy_ticks;
|
|
unlock(*r.mutex);
|
|
case .POINTER;
|
|
if r.options.cursor && have_image {
|
|
changed = true;
|
|
lock(*r.mutex); r.stats.pointer_moves += 1; unlock(*r.mutex);
|
|
}
|
|
}
|
|
|
|
// Paused: the source is still drained, so it never stalls, but nothing is sent. On resuming, the
|
|
// screen as it is now is sent at once.
|
|
now = ticks_now();
|
|
if atomic_read(*r.paused) {
|
|
was_paused = true;
|
|
lock(*r.mutex); r.stats.paused_seconds = cast(float64) (atomic_read(*r.paused_ticks) + now - atomic_read(*r.pause_began)) / r.frequency; unlock(*r.mutex);
|
|
continue;
|
|
}
|
|
if was_paused {
|
|
was_paused = false;
|
|
changed = true;
|
|
turn = now;
|
|
lock(*r.mutex); r.stats.paused_seconds = cast(float64) atomic_read(*r.paused_ticks) / r.frequency; unlock(*r.mutex);
|
|
}
|
|
if !have_image continue;
|
|
repeat := !changed && now - last_emit >= r.frequency;
|
|
if !(changed && now >= turn - early) && !repeat continue;
|
|
turn = max(turn + period, now - period); // after a quiet spell, no burst to catch up
|
|
|
|
// Copied to staging now; read back two frames on, when the GPU is done with it.
|
|
slot := emits % RECORD_STAGING;
|
|
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
|
|
ID3D11DeviceContext_CopyResource(gpu.device_context, gpu.staging[slot], gpu.latest);
|
|
staged[slot] = .{pending = true, ticks = record_timeline(r, now)};
|
|
if !r.using_wgc staged[slot].pointer, staged[slot].pointer_visible = r.dxgi.pointer_position, r.dxgi.pointer_visible;
|
|
emits += 1;
|
|
last_emit = now;
|
|
changed = false;
|
|
lock(*r.mutex);
|
|
r.stats.emitted += 1;
|
|
if repeat r.stats.repeats += 1;
|
|
unlock(*r.mutex);
|
|
}
|
|
|
|
// What is still staged, oldest first.
|
|
for 0..RECORD_STAGING - 1 {
|
|
slot := (emits + it) % RECORD_STAGING;
|
|
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
|
|
}
|
|
atomic_swap(*r.capture_ended, true);
|
|
signal(*r.ready);
|
|
return 0;
|
|
}
|
|
|
|
// Copies a staging texture into a free frame, draws the cursor (Desktop Duplication leaves it out)
|
|
// and queues it.
|
|
record_read_back :: (r: *Recorder, gpu: *Capture_Gpu, slot: s64, staged: Staged) {
|
|
t := ticks_now();
|
|
lock(*r.mutex);
|
|
if !r.free_count { // every frame is waiting: drop the oldest
|
|
r.free[r.free_count] = r.queue[0];
|
|
r.free_count += 1;
|
|
for 1..r.queue_count - 1 r.queue[it - 1] = r.queue[it];
|
|
r.queue_count -= 1;
|
|
r.stats.dropped += 1;
|
|
}
|
|
r.free_count -= 1;
|
|
index := r.free[r.free_count];
|
|
unlock(*r.mutex);
|
|
frame := *r.frames[index];
|
|
|
|
mapped: D3D11_MAPPED_SUBRESOURCE;
|
|
hr := ID3D11DeviceContext_Map(gpu.device_context, gpu.staging[slot], 0, .READ, 0, *mapped);
|
|
if FAILED(hr) {
|
|
lock(*r.mutex); r.free[r.free_count] = index; r.free_count += 1; unlock(*r.mutex);
|
|
record_fail(r, hresult_text("reading the frame back", hr));
|
|
return;
|
|
}
|
|
row := cast(s64) r.width * 4;
|
|
for y: 0..r.height - 1 memcpy(frame.pixels.data + y * row, cast(*u8) mapped.pData + cast(s64) y * mapped.RowPitch, row);
|
|
ID3D11DeviceContext_Unmap(gpu.device_context, gpu.staging[slot], 0);
|
|
if r.options.cursor && staged.pointer_visible {
|
|
cursor_draw(frame.pixels.data, r.width, r.height, *r.dxgi.shape,
|
|
staged.pointer.x - cast(s32) gpu.box.left, staged.pointer.y - cast(s32) gpu.box.top);
|
|
}
|
|
frame.ticks = staged.ticks;
|
|
|
|
lock(*r.mutex);
|
|
r.queue[r.queue_count] = index;
|
|
r.queue_count += 1;
|
|
memcpy(r.last_frame.data, frame.pixels.data, frame.pixels.count);
|
|
r.stats.readback_ticks += ticks_now() - t;
|
|
unlock(*r.mutex);
|
|
signal(*r.ready);
|
|
}
|
|
|
|
record_encoder_proc :: (thread: *Thread) -> s64 {
|
|
r := cast(*Recorder) thread.data;
|
|
while true {
|
|
wait_for(*r.ready, 100);
|
|
lock(*r.mutex);
|
|
index: s32 = -1;
|
|
if r.queue_count {
|
|
index = r.queue[0];
|
|
for 1..r.queue_count - 1 r.queue[it - 1] = r.queue[it];
|
|
r.queue_count -= 1;
|
|
}
|
|
unlock(*r.mutex);
|
|
if index < 0 {
|
|
if atomic_read(*r.capture_ended) break;
|
|
continue;
|
|
}
|
|
|
|
frame := *r.frames[index];
|
|
pts := max(frame.ticks, 0) * 90000 / r.frequency; // frame.ticks is on the timeline already
|
|
t := ticks_now();
|
|
ok, error := encoder_write(*r.encoder, frame.pixels.data, pts);
|
|
|
|
lock(*r.mutex);
|
|
r.free[r.free_count] = index;
|
|
r.free_count += 1;
|
|
r.stats.encode_ticks += ticks_now() - t;
|
|
if ok r.stats.written += 1;
|
|
r.stats.bytes = r.encoder.bytes;
|
|
unlock(*r.mutex);
|
|
if !ok {
|
|
record_fail(r, error);
|
|
// Frames still come until the capture thread sees it; they are let go.
|
|
while !atomic_read(*r.capture_ended) {
|
|
wait_for(*r.ready, 100);
|
|
lock(*r.mutex);
|
|
for 0..r.queue_count - 1 { r.free[r.free_count] = r.queue[it]; r.free_count += 1; }
|
|
r.queue_count = 0;
|
|
unlock(*r.mutex);
|
|
}
|
|
break;
|
|
}
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
// The audio thread: opens the sources asked for, waits for the encoder, then every 10 ms takes their
|
|
// packets into the mix and encodes what every source has covered.
|
|
record_audio_proc :: (thread: *Thread) -> s64 {
|
|
r := cast(*Recorder) thread.data;
|
|
audio_thread_start();
|
|
defer audio_thread_end();
|
|
|
|
kinds: [2] Audio_Source_Kind = .[.SYSTEM, .MICROPHONE];
|
|
wanted: [2] bool = .[r.options.system_audio, r.options.microphone];
|
|
names, errors: [..] string;
|
|
names.allocator, errors.allocator = temp, temp;
|
|
for 0..1 {
|
|
if !wanted[it] continue;
|
|
s := *r.audio_sources[it];
|
|
ok, error := audio_open(s, kinds[it]);
|
|
what := ifx kinds[it] == .SYSTEM then "system sound" else "microphone";
|
|
if ok { r.audio_open = true; array_add(*names, s.name); }
|
|
else { audio_close(s); array_add(*errors, tprint("%: %", what, error)); log_error("[record] No %: %", what, error); }
|
|
}
|
|
r.stats.audio = join(..names, "; ");
|
|
r.stats.audio_errors = join(..errors, "; ");
|
|
signal(*r.audio_opened);
|
|
wait_for(*r.audio_go);
|
|
if !r.audio_open || !r.encoder.audio || atomic_read(*r.stopping) {
|
|
for * r.audio_sources audio_free(it);
|
|
return 0;
|
|
}
|
|
|
|
mix: Audio_Mix;
|
|
mix.frame_size = r.encoder.audio_frame_size;
|
|
defer array_free(mix.samples);
|
|
while true {
|
|
stopping := atomic_read(*r.stopping);
|
|
for * source, index: r.audio_sources {
|
|
if !wanted[index] continue;
|
|
if !source.open { // lost: opened again a second on, from the default device then
|
|
if source.lost_at && ticks_now() - source.lost_at >= r.frequency && !stopping {
|
|
if audio_open(source, kinds[index]) log("[record] % is back: %.", ifx kinds[index] == .SYSTEM then "System sound" else "The microphone", source.name);
|
|
else { audio_close(source); source.lost_at = ticks_now(); }
|
|
}
|
|
continue;
|
|
}
|
|
packets, lost := audio_take(source, r.frequency);
|
|
for packets if !atomic_read(*r.paused) audio_mix_add(*mix, r, index, it);
|
|
if lost {
|
|
log_error("[record] % was lost (unplugged, or the default device changed); trying again.", source.name);
|
|
audio_close(source);
|
|
source.lost_at = ticks_now();
|
|
}
|
|
}
|
|
ok, error := audio_mix_encode(*mix, r, final = stopping);
|
|
if !ok { record_fail(r, error); break; }
|
|
if stopping break;
|
|
reset_temporary_storage();
|
|
sleep_milliseconds(10);
|
|
}
|
|
for * r.audio_sources audio_free(it);
|
|
return 0;
|
|
}
|
|
|
|
// The sound so far, from its base: one AAC frame's worth is encoded at a time.
|
|
Audio_Mix :: struct {
|
|
samples: [..] float; // interleaved stereo, from base on
|
|
base: s64; // the frame (in samples, on the timeline) samples[0] is
|
|
next: [2] s64; // each source's next frame, so packets that follow on are not moved by jitter
|
|
started: [2] bool;
|
|
frame_size: s32;
|
|
}
|
|
|
|
AUDIO_SLACK :: AUDIO_RATE / 5; // 200 ms: how late a source's packets may come and still be in the mix
|
|
AUDIO_FOLLOW_ON :: AUDIO_RATE / 33; // 30 ms: a packet that close to where its source left off follows on
|
|
|
|
// A packet into the mix at its place on the timeline, added to what the other source put there.
|
|
audio_mix_add :: (mix: *Audio_Mix, r: *Recorder, source: s64, packet: Audio_Packet) {
|
|
at := record_timeline(r, packet.ticks) * AUDIO_RATE / r.frequency;
|
|
if mix.started[source] && abs(at - mix.next[source]) <= AUDIO_FOLLOW_ON at = mix.next[source];
|
|
mix.started[source] = true;
|
|
mix.next[source] = at + packet.frames;
|
|
end := (at + packet.frames - mix.base) * AUDIO_CHANNELS;
|
|
if end > mix.samples.count {
|
|
old := mix.samples.count;
|
|
array_resize(*mix.samples, end, initialize = false);
|
|
for old..end - 1 mix.samples[it] = 0;
|
|
}
|
|
if packet.silent || !packet.samples.count return;
|
|
for frame: max(at, mix.base)..at + packet.frames - 1 {
|
|
into := (frame - mix.base) * AUDIO_CHANNELS;
|
|
from := (frame - at) * AUDIO_CHANNELS;
|
|
mix.samples[into] += packet.samples[from];
|
|
mix.samples[into + 1] += packet.samples[from + 1];
|
|
}
|
|
}
|
|
|
|
// Encodes the frames every open source has covered: up to where each has sent, or, for one that has
|
|
// sent nothing lately (silence), to AUDIO_SLACK ago. final: everything sent.
|
|
audio_mix_encode :: (mix: *Audio_Mix, r: *Recorder, final: bool) -> ok: bool, error: string {
|
|
// Paused, the timeline stands still where the pause began.
|
|
now := ifx atomic_read(*r.paused) then atomic_read(*r.pause_began) else ticks_now();
|
|
now_frame := record_timeline(r, now) * AUDIO_RATE / r.frequency;
|
|
ready := now_frame - AUDIO_SLACK;
|
|
for source, index: r.audio_sources {
|
|
if !source.open continue;
|
|
covered := max(mix.next[index], now_frame - AUDIO_SLACK);
|
|
ready = min(ready, covered);
|
|
}
|
|
if final { ready = mix.base; for mix.next ready = max(ready, it); }
|
|
|
|
size := mix.frame_size;
|
|
frame := NewArray(size * AUDIO_CHANNELS, float, initialized = false,, temp);
|
|
done: s64;
|
|
while mix.base + size <= ready {
|
|
for i: 0..frame.count - 1 {
|
|
at := done * AUDIO_CHANNELS + i;
|
|
value := ifx at < mix.samples.count then mix.samples[at] else 0.0;
|
|
frame[i] = clamp(value, -1.0, 1.0);
|
|
}
|
|
ok, error := encoder_write_audio(*r.encoder, frame.data, mix.base);
|
|
if !ok return false, error;
|
|
mix.base += size;
|
|
done += size;
|
|
lock(*r.mutex); r.stats.audio_frames += 1; unlock(*r.mutex);
|
|
}
|
|
// What was encoded leaves the front.
|
|
gone := min(done * AUDIO_CHANNELS, mix.samples.count);
|
|
if gone {
|
|
memcpy(mix.samples.data, mix.samples.data + gone, (mix.samples.count - gone) * size_of(float));
|
|
mix.samples.count -= gone;
|
|
}
|
|
return true, "";
|
|
}
|
|
|
|
// Ends the recording from one of its threads; the first reason is kept.
|
|
record_fail :: (r: *Recorder, error: string) {
|
|
log_error("[record] %", error);
|
|
lock(*r.mutex);
|
|
if !r.error r.error = copy_string(error);
|
|
unlock(*r.mutex);
|
|
atomic_swap(*r.stopping, true);
|
|
}
|
|
|
|
rect_inside :: (inner: RECT, outer: RECT) -> bool {
|
|
return inner.left >= outer.left && inner.top >= outer.top && inner.right <= outer.right && inner.bottom <= outer.bottom;
|
|
}
|
|
|
|
#scope_export
|
|
|
|
#load "capture_gpu.jai";
|
|
#load "dxgi_source.jai";
|
|
#load "wgc_source.jai";
|
|
#load "encoder.jai";
|
|
#load "remux.jai";
|
|
#load "audio_source.jai";
|
|
#load "cursor.jai";
|
|
|
|
#import "Basic";
|
|
#import "String";
|
|
#import "Atomics";
|
|
#import "Math";
|
|
#import "Thread";
|
|
#import "Windows";
|
|
#import "Windows_Utf8";
|
|
#import "dxgi";
|
|
#import "d3d11";
|
|
#import "ffmpeg";
|