Compare commits
4 Commits
59c5b8dd62
...
4acde4d7a6
| Author | SHA1 | Date | |
|---|---|---|---|
| 4acde4d7a6 | |||
| 3f5d1ea124 | |||
| 72f2754226 | |||
| e0eaf5642b |
22
README.md
22
README.md
@ -187,10 +187,14 @@ register the key, the Event Log says so; choose another, such as `Ctrl-Shift-S
|
||||
go of it starts nothing; drag again.
|
||||
- Escape or a right click cancels.
|
||||
3. **Recording starts** at once, at up to 30 frames a second, cursor included. A frame is written
|
||||
only when the screen changes, so a still screen costs almost nothing.
|
||||
only when the screen changes, so a still screen costs almost nothing. **Sound:** what the
|
||||
computer plays is recorded with it, and the microphone once you turn it on. The toolbar's
|
||||
**System sound** and **Mic** buttons turn each on and off, mid-recording too, and their state is
|
||||
kept for the next recording.
|
||||
- **A red border** is drawn just outside the region.
|
||||
- **A toolbar** beside it shows the time, the size and any frames dropped, with **Pause** (the time
|
||||
paused is left out of the video), **Stop**, and **Discard**, which asks first.
|
||||
- **A toolbar** beside it shows the time, the size and any frames dropped, with **System sound**
|
||||
and **Mic** (each on or off), **Pause** (the time paused is left out of the video), **Stop**, and **Discard**, which asks
|
||||
first.
|
||||
- **Neither is in the recording.** Neither takes the focus from the window being recorded.
|
||||
4. **Ctrl+PrintScreen again**, or Stop, stops it. The video goes into **Screen Captures** beside the stills,
|
||||
as an H.264 MP4 in an `.mcap` with the window it was recorded from, and opens in the Capture window.
|
||||
@ -204,9 +208,15 @@ register the key, the Event Log says so; choose another, such as `Ctrl-Shift-S
|
||||
- A recording that stops by itself keeps what it has, and so does quitting while recording.
|
||||
- When the screen is lost for a moment (a UAC prompt, the lock screen, a resolution change), the
|
||||
recording carries on once it is back.
|
||||
- A recording a crash cut short is offered at the next start: into the library, or deleted.
|
||||
- **Settings:** `[video]` in the settings file: `fps` (30), `cursor` (true), and `capture`: `auto`
|
||||
(Desktop Duplication, else Windows.Graphics.Capture), `dxgi` or `wgc`.
|
||||
- A recording a crash cut short is offered at the next start: into the library, or deleted. It
|
||||
keeps all but its last two seconds or so.
|
||||
- **Settings:** `[video]` in the settings file:
|
||||
- `fps` (30) and `cursor` (true).
|
||||
- `capture`: `auto` (Desktop Duplication, else Windows.Graphics.Capture), `dxgi` or `wgc`.
|
||||
- `system_audio` (true): what the computer plays, from the default output device. The toolbar's
|
||||
System sound button sets it.
|
||||
- `microphone` (false): the default input device. The toolbar's Mic button sets it.
|
||||
- Sound is AAC at 160 kb/s. A device unplugged mid-recording is picked up again from the new default.
|
||||
- **PrintScreen while recording** takes the whole screen under the cursor at once, without freezing
|
||||
the screens, which would show in the recording.
|
||||
- **The key** is `record_region` in `[global_hotkeys]`.
|
||||
|
||||
@ -29,7 +29,7 @@ APP_NAME :: "mcap";
|
||||
APP_FOLDER_NAME :: "mcap";
|
||||
GIT_BRANCH :: "check";
|
||||
GIT_HASH :: "-";
|
||||
VERSION :: "0.1.6";
|
||||
VERSION :: "0.1.10";
|
||||
BUILD_DATE :: "-";
|
||||
DEBUG :: true;
|
||||
COMPILE_TESTS :: true;
|
||||
|
||||
@ -8,7 +8,7 @@ APP_NAME :: "mcap";
|
||||
APP_FOLDER_NAME :: "mcap";
|
||||
APP_BUNDLE_IDENTIFIER :: "com.example.mcap";
|
||||
|
||||
VERSION :: "0.1.6";
|
||||
VERSION :: "0.1.10";
|
||||
// Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated
|
||||
// builds never write over the ones you run by hand.
|
||||
build_directory := "bin";
|
||||
|
||||
@ -120,8 +120,10 @@ Format: "PNG"
|
||||
|
||||
## Screen recordings
|
||||
|
||||
A recording made with the mcap app's Ctrl+PrintScreen is a video: its original is the H.264 MP4 as
|
||||
recorded, and its rendered image (kind 2) a frame from it as a PNG, as for an imported video. Its
|
||||
A recording made with the mcap app's Ctrl+PrintScreen is a video: its original is the H.264 MP4, and
|
||||
its rendered image (kind 2) a frame from it as a PNG, as for an imported video. It is recorded as a
|
||||
fragmented MP4, so a crash still leaves one that plays, and stored as a regular one, its frames copied
|
||||
as they are: one header first, with its length and the index of every frame. Its
|
||||
metadata.json is a still's. Its capture.json has a still's keys, with Format "MP4", and these:
|
||||
|
||||
```
|
||||
@ -134,6 +136,9 @@ Capture: "DXGI DuplicateOutput1 on NVIDIA GeForce RTX 5080 Laptop GPU"
|
||||
or "Windows.Graphics.Capture on ..., border off"
|
||||
DroppedFrames: 0 frames the encoder fell too far behind for
|
||||
Paused: 10.2 seconds paused, left out of the video; only when it was paused
|
||||
Audio: true whether it has sound (AAC)
|
||||
AudioSources: "SAMSUNG (NVIDIA High Definition Audio); Microphone Array (Realtek(R) Audio)"
|
||||
the devices heard; only when it has sound
|
||||
```
|
||||
|
||||
A recording a crash cut short, put in the library at the next start, has no window, Encoder or
|
||||
|
||||
263
modules/Screen_Record/audio_source.jai
Normal file
263
modules/Screen_Record/audio_source.jai
Normal file
@ -0,0 +1,263 @@
|
||||
// Sound for a recording, with WASAPI: what the computer plays (the default output device, in loopback)
|
||||
// and the microphone (the default input device). Each is asked for in shared mode as 48 kHz stereo
|
||||
// float, which Windows converts any device's own format to (AUTOCONVERTPCM), so nothing here
|
||||
// resamples. Where a device will not convert, its own format is taken if it is 48 kHz float, and its
|
||||
// channels are made stereo here.
|
||||
//
|
||||
// Each packet comes with the QueryPerformanceCounter time its first sample was heard, which places it
|
||||
// on the recording's timeline (module.jai's audio thread mixes them). Loopback sends nothing while
|
||||
// nothing plays: those gaps are silence. A device unplugged mid-recording (a headset) is opened
|
||||
// again, from the new default, once a second.
|
||||
|
||||
AUDIO_RATE :: 48000;
|
||||
AUDIO_CHANNELS :: 2;
|
||||
|
||||
Audio_Source_Kind :: enum u8 { SYSTEM; MICROPHONE; }
|
||||
|
||||
Audio_Source :: struct {
|
||||
kind: Audio_Source_Kind;
|
||||
name: string; // "Speakers (Realtek(R) Audio)"; heap
|
||||
client: *Audio_Client;
|
||||
capture: *Audio_Capture_Client;
|
||||
channels: s32; // as the device gives them: 2, unless it would not convert
|
||||
open: bool;
|
||||
lost_at: s64; // when it was lost (ticks), to open it again a second on; 0 while open
|
||||
error: string; // why it could not be opened; heap
|
||||
}
|
||||
|
||||
Audio_Packet :: struct {
|
||||
samples: [] float; // interleaved stereo, AUDIO_CHANNELS a frame; empty for silence
|
||||
frames: s64;
|
||||
ticks: s64; // when its first frame was heard, in QueryPerformanceCounter ticks
|
||||
silent: bool;
|
||||
}
|
||||
|
||||
// The default device of its kind, started. False with why: none, or it will not open.
|
||||
audio_open :: (s: *Audio_Source, kind: Audio_Source_Kind) -> ok: bool, error: string {
|
||||
s.kind = kind;
|
||||
enumerator: *Device_Enumerator;
|
||||
hr := audio_CoCreateInstance(*CLSID_MM_DEVICE_ENUMERATOR, null, CLSCTX_ALL, *IID_MM_DEVICE_ENUMERATOR, xx *enumerator);
|
||||
if FAILED(hr) return false, hresult_text("the audio device list", hr);
|
||||
defer com_release(*enumerator);
|
||||
device: *Audio_Device;
|
||||
flow: s32 = ifx kind == .SYSTEM then E_RENDER else E_CAPTURE;
|
||||
hr = enumerator.vtable.GetDefaultAudioEndpoint(enumerator, flow, E_CONSOLE, *device);
|
||||
if FAILED(hr) return false, ifx kind == .SYSTEM then "no sound output device" else "no microphone";
|
||||
defer com_release(*device);
|
||||
free(s.name);
|
||||
s.name = copy_string(audio_device_name(device));
|
||||
|
||||
hr = device.vtable.Activate(device, *IID_AUDIO_CLIENT, CLSCTX_ALL, null, xx *s.client);
|
||||
if FAILED(hr) return false, hresult_text(tprint("% (%)", s.name, "IAudioClient"), hr);
|
||||
|
||||
flags: u32 = AUTOCONVERTPCM | SRC_DEFAULT_QUALITY;
|
||||
if kind == .SYSTEM flags |= STREAMFLAGS_LOOPBACK;
|
||||
wanted := WAVEFORMATEX.{wFormatTag = WAVE_FORMAT_IEEE_FLOAT, nChannels = AUDIO_CHANNELS, nSamplesPerSec = AUDIO_RATE,
|
||||
nAvgBytesPerSec = AUDIO_RATE * AUDIO_CHANNELS * 4, nBlockAlign = AUDIO_CHANNELS * 4, wBitsPerSample = 32};
|
||||
BUFFER :: 2_000_000; // 200 ms, in 100 ns: room for the audio thread being late
|
||||
hr = s.client.vtable.Initialize(s.client, SHAREMODE_SHARED, flags, BUFFER, 0, *wanted, null);
|
||||
s.channels = AUDIO_CHANNELS;
|
||||
if FAILED(hr) {
|
||||
// No conversion here: the device's own format, if it is 48 kHz float, its channels made stereo after.
|
||||
mix: *WAVEFORMATEX;
|
||||
if FAILED(s.client.vtable.GetMixFormat(s.client, *mix)) return false, hresult_text(tprint("% will not give 48 kHz stereo", s.name), hr);
|
||||
defer audio_CoTaskMemFree(mix);
|
||||
is_float := mix.wFormatTag == WAVE_FORMAT_IEEE_FLOAT || (mix.wFormatTag == WAVE_FORMAT_EXTENSIBLE && mix.wBitsPerSample == 32);
|
||||
if !is_float || mix.nSamplesPerSec != AUDIO_RATE
|
||||
return false, tprint("% gives % Hz %, not 48 kHz float, and will not convert", s.name, mix.nSamplesPerSec, ifx is_float then "float" else "PCM");
|
||||
com_release(*s.client);
|
||||
hr = device.vtable.Activate(device, *IID_AUDIO_CLIENT, CLSCTX_ALL, null, xx *s.client);
|
||||
if FAILED(hr) return false, hresult_text(s.name, hr);
|
||||
hr = s.client.vtable.Initialize(s.client, SHAREMODE_SHARED, flags & ~(AUTOCONVERTPCM | SRC_DEFAULT_QUALITY), BUFFER, 0, mix, null);
|
||||
if FAILED(hr) return false, hresult_text(tprint("% (Initialize)", s.name), hr);
|
||||
s.channels = mix.nChannels;
|
||||
}
|
||||
hr = s.client.vtable.GetService(s.client, *IID_AUDIO_CAPTURE_CLIENT, xx *s.capture);
|
||||
if FAILED(hr) return false, hresult_text(tprint("% (IAudioCaptureClient)", s.name), hr);
|
||||
hr = s.client.vtable.Start(s.client);
|
||||
if FAILED(hr) return false, hresult_text(tprint("% (Start)", s.name), hr);
|
||||
s.open = true;
|
||||
s.lost_at = 0;
|
||||
return true, "";
|
||||
}
|
||||
|
||||
// The packets waiting, each as stereo float (temp), oldest first. lost: the device went away
|
||||
// (unplugged, or the default changed): audio_close, and open it again later.
|
||||
audio_take :: (s: *Audio_Source, frequency: s64) -> [] Audio_Packet, lost: bool {
|
||||
packets: [..] Audio_Packet;
|
||||
packets.allocator = temp;
|
||||
while true {
|
||||
next: u32;
|
||||
hr := s.capture.vtable.GetNextPacketSize(s.capture, *next);
|
||||
if hresult_is(hr, AUDCLNT_E_DEVICE_INVALIDATED) return packets, true;
|
||||
if FAILED(hr) || !next break;
|
||||
data: *u8;
|
||||
frames, flags: u32;
|
||||
device_position, qpc_position: u64; // qpc_position: QueryPerformanceCounter time, in 100 ns
|
||||
hr = s.capture.vtable.GetBuffer(s.capture, *data, *frames, *flags, *device_position, *qpc_position);
|
||||
if hresult_is(hr, AUDCLNT_E_DEVICE_INVALIDATED) return packets, true;
|
||||
if FAILED(hr) || hresult_is(hr, AUDCLNT_S_BUFFER_EMPTY) break;
|
||||
|
||||
packet: Audio_Packet;
|
||||
packet.frames = frames;
|
||||
packet.ticks = cast(s64) (cast(float64) qpc_position * frequency / 10_000_000.0);
|
||||
packet.silent = (flags & BUFFERFLAGS_SILENT) != 0;
|
||||
if !packet.silent && frames {
|
||||
packet.samples = NewArray(cast(s64) frames * AUDIO_CHANNELS, float, initialized = false,, temp);
|
||||
input := cast(*float) data;
|
||||
if s.channels == AUDIO_CHANNELS {
|
||||
memcpy(packet.samples.data, input, cast(s64) frames * AUDIO_CHANNELS * size_of(float));
|
||||
} else {
|
||||
// Mono to both sides; more channels than two: the first two (front left and right).
|
||||
channels := cast(s64) s.channels;
|
||||
for f: 0..cast(s64) frames - 1 {
|
||||
left := input[f * channels];
|
||||
right := ifx channels > 1 then input[f * channels + 1] else left;
|
||||
packet.samples[f * 2] = left;
|
||||
packet.samples[f * 2 + 1] = right;
|
||||
}
|
||||
}
|
||||
}
|
||||
s.capture.vtable.ReleaseBuffer(s.capture, frames);
|
||||
array_add(*packets, packet);
|
||||
}
|
||||
return packets, false;
|
||||
}
|
||||
|
||||
audio_close :: (s: *Audio_Source) {
|
||||
if s.client && s.open s.client.vtable.Stop(s.client);
|
||||
com_release(*s.capture);
|
||||
com_release(*s.client);
|
||||
s.open = false;
|
||||
}
|
||||
|
||||
audio_free :: (s: *Audio_Source) {
|
||||
audio_close(s);
|
||||
free(s.name);
|
||||
free(s.error);
|
||||
s.name, s.error = "", "";
|
||||
}
|
||||
|
||||
// COM on this thread, in the multithreaded apartment, as WASAPI's objects are used from it only.
|
||||
audio_thread_start :: () {
|
||||
audio_CoInitializeEx(null, COINIT_MULTITHREADED);
|
||||
}
|
||||
|
||||
audio_thread_end :: () {
|
||||
audio_CoUninitialize();
|
||||
}
|
||||
|
||||
#scope_file
|
||||
|
||||
audio_device_name :: (device: *Audio_Device) -> string {
|
||||
store: *Property_Store;
|
||||
if FAILED(device.vtable.OpenPropertyStore(device, STGM_READ, *store)) return "";
|
||||
defer com_release(*store);
|
||||
value: PROPVARIANT;
|
||||
if FAILED(store.vtable.GetValue(store, *PKEY_DEVICE_FRIENDLY_NAME, *value)) return "";
|
||||
defer audio_PropVariantClear(*value);
|
||||
VT_LPWSTR :: 31;
|
||||
if value.vt != VT_LPWSTR || !value.pointer return "";
|
||||
name, ok := wide_to_utf8(cast(*u16) value.pointer,, temp);
|
||||
return ifx ok then name else "";
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- bindings
|
||||
|
||||
ole32_dll :: #library,system "ole32";
|
||||
|
||||
audio_CoInitializeEx :: (reserved: *void, flags: u32) -> HRESULT #foreign ole32_dll "CoInitializeEx";
|
||||
audio_CoUninitialize :: () #foreign ole32_dll "CoUninitialize";
|
||||
audio_CoCreateInstance :: (clsid: *GUID, outer: *void, class_context: u32, iid: *GUID, out: **void) -> HRESULT #foreign ole32_dll "CoCreateInstance";
|
||||
audio_CoTaskMemFree :: (memory: *void) #foreign ole32_dll "CoTaskMemFree";
|
||||
audio_PropVariantClear :: (value: *PROPVARIANT) -> HRESULT #foreign ole32_dll "PropVariantClear";
|
||||
|
||||
COINIT_MULTITHREADED : u32 : 0;
|
||||
CLSCTX_ALL : u32 : 0x17;
|
||||
E_RENDER : s32 : 0;
|
||||
E_CAPTURE : s32 : 1;
|
||||
E_CONSOLE : s32 : 0;
|
||||
STGM_READ : u32 : 0;
|
||||
SHAREMODE_SHARED : s32 : 0;
|
||||
STREAMFLAGS_LOOPBACK : u32 : 0x00020000;
|
||||
AUTOCONVERTPCM : u32 : 0x80000000;
|
||||
SRC_DEFAULT_QUALITY : u32 : 0x08000000;
|
||||
BUFFERFLAGS_SILENT : u32 : 0x2;
|
||||
WAVE_FORMAT_IEEE_FLOAT : u16 : 3;
|
||||
WAVE_FORMAT_EXTENSIBLE : u16 : 0xFFFE;
|
||||
AUDCLNT_S_BUFFER_EMPTY :: 0x08890001;
|
||||
AUDCLNT_E_DEVICE_INVALIDATED :: 0x88890004;
|
||||
|
||||
CLSID_MM_DEVICE_ENUMERATOR := #run guid("BCDE0395-E52F-467C-8E3D-C4579291692E");
|
||||
IID_MM_DEVICE_ENUMERATOR := #run guid("A95664D2-9614-4F35-A746-DE8DB63617E6");
|
||||
IID_AUDIO_CLIENT := #run guid("1CB9AD4C-DBFA-4C32-B178-C2F568A703B2");
|
||||
IID_AUDIO_CAPTURE_CLIENT := #run guid("C8ADBD64-E71E-48A0-A4DE-185C395CD317");
|
||||
|
||||
PROPERTYKEY :: struct { fmtid: GUID; pid: u32; }
|
||||
PKEY_DEVICE_FRIENDLY_NAME := PROPERTYKEY.{#run guid("A45C254E-DF1C-4EFD-8020-67D146A850E0"), 14};
|
||||
|
||||
// A PROPVARIANT, as far as a string in it: its type, then (after 6 reserved bytes) the pointer.
|
||||
PROPVARIANT :: struct {
|
||||
vt: u16;
|
||||
reserved: [3] u16;
|
||||
pointer: *void;
|
||||
padding: *void;
|
||||
}
|
||||
|
||||
WAVEFORMATEX :: struct {
|
||||
wFormatTag: u16;
|
||||
nChannels: u16;
|
||||
nSamplesPerSec: u32;
|
||||
nAvgBytesPerSec: u32;
|
||||
nBlockAlign: u16;
|
||||
wBitsPerSample: u16;
|
||||
cbSize: u16;
|
||||
}
|
||||
|
||||
Unknown_Methods :: struct {
|
||||
QueryInterface: (this: *void, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
AddRef: (this: *void) -> u32 #c_call;
|
||||
Release: (this: *void) -> u32 #c_call;
|
||||
}
|
||||
|
||||
Device_Enumerator :: struct { vtable: *struct {
|
||||
using base: Unknown_Methods;
|
||||
EnumAudioEndpoints: *void;
|
||||
GetDefaultAudioEndpoint: (this: *Device_Enumerator, flow: s32, role: s32, out: **Audio_Device) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Audio_Device :: struct { vtable: *struct {
|
||||
using base: Unknown_Methods;
|
||||
Activate: (this: *Audio_Device, iid: *GUID, class_context: u32, parameters: *void, out: **void) -> HRESULT #c_call;
|
||||
OpenPropertyStore: (this: *Audio_Device, access: u32, out: **Property_Store) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Property_Store :: struct { vtable: *struct {
|
||||
using base: Unknown_Methods;
|
||||
GetCount: *void;
|
||||
GetAt: *void;
|
||||
GetValue: (this: *Property_Store, key: *PROPERTYKEY, out: *PROPVARIANT) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Audio_Client :: struct { vtable: *struct {
|
||||
using base: Unknown_Methods;
|
||||
Initialize: (this: *Audio_Client, share_mode: s32, flags: u32, buffer_duration: s64, periodicity: s64, format: *WAVEFORMATEX, session: *GUID) -> HRESULT #c_call;
|
||||
GetBufferSize: *void;
|
||||
GetStreamLatency: *void;
|
||||
GetCurrentPadding: *void;
|
||||
IsFormatSupported: *void;
|
||||
GetMixFormat: (this: *Audio_Client, out: **WAVEFORMATEX) -> HRESULT #c_call;
|
||||
GetDevicePeriod: *void;
|
||||
Start: (this: *Audio_Client) -> HRESULT #c_call;
|
||||
Stop: (this: *Audio_Client) -> HRESULT #c_call;
|
||||
Reset: *void;
|
||||
SetEventHandle: *void;
|
||||
GetService: (this: *Audio_Client, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Audio_Capture_Client :: struct { vtable: *struct {
|
||||
using base: Unknown_Methods;
|
||||
GetBuffer: (this: *Audio_Capture_Client, data: **u8, frames: *u32, flags: *u32, device_position: *u64, qpc_position: *u64) -> HRESULT #c_call;
|
||||
ReleaseBuffer: (this: *Audio_Capture_Client, frames: u32) -> HRESULT #c_call;
|
||||
GetNextPacketSize: (this: *Audio_Capture_Client, out: *u32) -> HRESULT #c_call;
|
||||
}; }
|
||||
@ -78,6 +78,26 @@ gpu_thread_dpi_aware :: () {
|
||||
user32_dll :: #library,system "user32";
|
||||
SetThreadDpiAwarenessContext :: (value: HANDLE) -> HANDLE #foreign user32_dll;
|
||||
|
||||
// A GUID from its text, "A95664D2-9614-4F35-A746-DE8DB63617E6": for #run.
|
||||
guid :: (text: string) -> GUID {
|
||||
hex :: (s: string) -> u64 {
|
||||
value: u64;
|
||||
for 0..s.count - 1 {
|
||||
c := s[it];
|
||||
digit: u64 = ifx c >= #char "a" then cast(u64) (c - #char "a" + 10) else ifx c >= #char "A" then cast(u64) (c - #char "A" + 10) else cast(u64) (c - #char "0");
|
||||
value = value * 16 + digit;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
g: GUID;
|
||||
g.Data1 = cast(u32) hex(slice(text, 0, 8));
|
||||
g.Data2 = cast(u16) hex(slice(text, 9, 4));
|
||||
g.Data3 = cast(u16) hex(slice(text, 14, 4));
|
||||
for 0..1 g.Data4[it] = cast(u8) hex(slice(text, 19 + it * 2, 2));
|
||||
for 0..5 g.Data4[it + 2] = cast(u8) hex(slice(text, 24 + it * 2, 2));
|
||||
return g;
|
||||
}
|
||||
|
||||
com_release :: (p: *$T) {
|
||||
if p.* { IUnknown_Release(cast(*IUnknown) p.*); p.* = null; }
|
||||
}
|
||||
|
||||
@ -4,10 +4,17 @@
|
||||
// BGRA as it is and converts it on the GPU; the others are given NV12 or YUV 4:2:0, converted with
|
||||
// swscale (BT.709, limited range, as tagged).
|
||||
//
|
||||
// The file is written fragmented (a fragment each keyframe, every 2 s), so one cut short by a crash
|
||||
// still plays up to its last fragment.
|
||||
// With sound, an AAC stream (FFmpeg's own encoder, 160 kb/s, 48 kHz stereo) beside it, written from the
|
||||
// audio thread: the two streams' packets meet in the file under a lock (mux).
|
||||
//
|
||||
// The file is written fragmented, a fragment at each keyframe, so one cut short by a crash still plays
|
||||
// up to its last fragment. Frames are written only when the screen changes, so a count of frames can
|
||||
// span any time: a keyframe is made every RECORD_KEYFRAME_SECONDS of time instead, which bounds what a
|
||||
// crash loses. remux.jai makes the file a regular MP4 once it is done.
|
||||
|
||||
RECORD_ENCODERS :: string.["h264_nvenc", "h264_amf", "h264_qsv", "h264_mf", "libx264"];
|
||||
RECORD_KEYFRAME_SECONDS :: 2;
|
||||
RECORD_AUDIO_BIT_RATE :: 160_000;
|
||||
|
||||
Encoder :: struct {
|
||||
format: *AVFormatContext;
|
||||
@ -20,13 +27,23 @@ Encoder :: struct {
|
||||
input: AVPixelFormat; // what it is given: BGR0 as captured, or converted to NV12 or YUV420P
|
||||
width, height: s32;
|
||||
last_pts: s64 = -1;
|
||||
last_key_pts: s64 = -1; // the last keyframe asked for
|
||||
header_written: bool;
|
||||
bytes: s64; // of packets written
|
||||
|
||||
// Sound, when asked for: AAC, written from the audio thread.
|
||||
audio: *AVCodecContext;
|
||||
audio_stream: *AVStream;
|
||||
audio_frame: *AVFrame;
|
||||
audio_packet: *AVPacket;
|
||||
audio_frame_size: s32; // samples a frame: AAC's 1024
|
||||
mux: Mutex; // the video and audio threads' packets into the file
|
||||
mux_ready: bool;
|
||||
}
|
||||
|
||||
// Opens the file and the first encoder that will take width x height (wanted alone, when given).
|
||||
// Failures to open an encoder are listed in tried (temp), for the log.
|
||||
encoder_open :: (e: *Encoder, path: string, width: s32, height: s32, fps: s32, wanted: string) -> ok: bool, error: string, tried: string {
|
||||
encoder_open :: (e: *Encoder, path: string, width: s32, height: s32, fps: s32, wanted: string, audio := false) -> ok: bool, error: string, tried: string {
|
||||
av_log_set_level(AV_LOG_ERROR);
|
||||
e.width, e.height = width, height;
|
||||
result := avformat_alloc_output_context2(*e.format, null, "mp4", temp_c_string(path));
|
||||
@ -51,6 +68,12 @@ encoder_open :: (e: *Encoder, path: string, width: s32, height: s32, fps: s32, w
|
||||
if !e.stream { encoder_free(e); return false, "FFmpeg cannot add a stream", tried_text; }
|
||||
avcodec_parameters_from_context(e.stream.codecpar, e.codec);
|
||||
e.stream.time_base = e.codec.time_base;
|
||||
if audio {
|
||||
audio_ok, audio_error := encoder_open_audio(e);
|
||||
if !audio_ok { encoder_free(e); return false, audio_error, tried_text; }
|
||||
}
|
||||
init(*e.mux, "record mux");
|
||||
e.mux_ready = true;
|
||||
|
||||
result = avio_open(*e.format.pb, temp_c_string(path), AVIO_FLAG_WRITE);
|
||||
if result < 0 { encoder_free(e); return false, av_error(tprint("FFmpeg cannot write %", path), result), tried_text; }
|
||||
@ -95,6 +118,10 @@ encoder_write :: (e: *Encoder, pixels: *u8, pts: s64) -> ok: bool, error: string
|
||||
sws_scale(e.sws, *source, *stride, 0, e.height, frame.data.data, frame.linesize.data);
|
||||
}
|
||||
frame.pts = pts_out;
|
||||
if e.last_key_pts < 0 || pts_out - e.last_key_pts >= RECORD_KEYFRAME_SECONDS * RECORD_TIME_BASE {
|
||||
frame.pict_type = .I; // an IDR, with forced-idr: a new fragment
|
||||
e.last_key_pts = pts_out;
|
||||
}
|
||||
|
||||
result := avcodec_send_frame(e.codec, frame);
|
||||
av_frame_unref(frame);
|
||||
@ -103,12 +130,36 @@ encoder_write :: (e: *Encoder, pixels: *u8, pts: s64) -> ok: bool, error: string
|
||||
return drained, error;
|
||||
}
|
||||
|
||||
// Flushes the encoder, finishes the file and closes it.
|
||||
// One AAC frame of sound: e.audio_frame_size frames of interleaved stereo float, at pts in samples.
|
||||
// From the audio thread.
|
||||
encoder_write_audio :: (e: *Encoder, samples: *float, pts: s64) -> ok: bool, error: string {
|
||||
frame := e.audio_frame;
|
||||
result := av_frame_make_writable(frame);
|
||||
if result < 0 return false, av_error("FFmpeg cannot make a sound frame", result);
|
||||
left := cast(*float) frame.data[0];
|
||||
right := cast(*float) frame.data[1];
|
||||
for 0..e.audio_frame_size - 1 {
|
||||
left[it] = samples[it * 2];
|
||||
right[it] = samples[it * 2 + 1];
|
||||
}
|
||||
frame.pts = pts;
|
||||
result = avcodec_send_frame(e.audio, frame);
|
||||
if result < 0 return false, av_error("the AAC encoder", result);
|
||||
drained, error := encoder_drain(e, e.audio, e.audio_stream, e.audio_packet);
|
||||
return drained, error;
|
||||
}
|
||||
|
||||
// Flushes the encoders, sound first, finishes the file and closes it.
|
||||
encoder_close :: (e: *Encoder) -> ok: bool, error: string {
|
||||
ok, error := true, "";
|
||||
if e.audio && e.header_written {
|
||||
avcodec_send_frame(e.audio, null);
|
||||
ok, error = encoder_drain(e, e.audio, e.audio_stream, e.audio_packet);
|
||||
}
|
||||
if e.codec && e.header_written {
|
||||
avcodec_send_frame(e.codec, null);
|
||||
ok, error = encoder_drain(e);
|
||||
video_ok, video_error := encoder_drain(e);
|
||||
if ok ok, error = video_ok, video_error;
|
||||
result := av_write_trailer(e.format);
|
||||
if ok && result < 0 ok, error = false, av_error("FFmpeg cannot finish the MP4", result);
|
||||
}
|
||||
@ -147,11 +198,11 @@ encoder_try :: (e: *Encoder, name: string, width: s32, height: s32, fps: s32) ->
|
||||
options: *AVDictionary;
|
||||
set :: (key: string, value: string) #expand { av_dict_set(*`options, temp_c_string(key), temp_c_string(value), 0); }
|
||||
if name == {
|
||||
case "h264_nvenc"; set("preset", "p4"); set("rc", "vbr"); set("cq", "23"); c.bit_rate = 0;
|
||||
case "h264_nvenc"; set("preset", "p4"); set("rc", "vbr"); set("cq", "23"); set("forced-idr", "1"); c.bit_rate = 0;
|
||||
case "h264_amf"; set("rc", "cqp"); set("qp_i", "22"); set("qp_p", "24"); set("quality", "balanced");
|
||||
case "h264_qsv"; set("global_quality", "23"); set("preset", "medium");
|
||||
case "h264_mf"; set("rate_control", "quality"); set("quality", "70"); set("scenario", "display_remoting");
|
||||
case "libx264"; set("preset", "veryfast"); set("crf", "23");
|
||||
case "libx264"; set("preset", "veryfast"); set("crf", "23"); set("forced-idr", "1");
|
||||
}
|
||||
result := avcodec_open2(c, codec, *options);
|
||||
av_dict_free(*options);
|
||||
@ -178,22 +229,64 @@ encoder_input_format :: (codec: *AVCodec) -> AVPixelFormat {
|
||||
return .NONE;
|
||||
}
|
||||
|
||||
encoder_drain :: (e: *Encoder) -> ok: bool, error: string {
|
||||
// The packets an encoder (the video's, unless another is given) has ready, into the file.
|
||||
encoder_drain :: (e: *Encoder, codec: *AVCodecContext = null, stream: *AVStream = null, packet: *AVPacket = null) -> ok: bool, error: string {
|
||||
AVERROR_EAGAIN :: -11;
|
||||
if !codec codec, stream, packet = e.codec, e.stream, e.packet;
|
||||
while true {
|
||||
result := avcodec_receive_packet(e.codec, e.packet);
|
||||
result := avcodec_receive_packet(codec, packet);
|
||||
if result == AVERROR_EAGAIN || result == cast,no_check(s32) AVERROR_EOF return true, "";
|
||||
if result < 0 return false, av_error(tprint("% failed", e.name), result);
|
||||
av_packet_rescale_ts(e.packet, e.codec.time_base, e.stream.time_base);
|
||||
e.packet.stream_index = e.stream.index;
|
||||
e.bytes += e.packet.size;
|
||||
result = av_interleaved_write_frame(e.format, e.packet); // takes the packet's data
|
||||
if result < 0 return false, av_error(tprint("% failed", ifx codec == e.audio then "the AAC encoder" else e.name), result);
|
||||
av_packet_rescale_ts(packet, codec.time_base, stream.time_base);
|
||||
packet.stream_index = stream.index;
|
||||
lock(*e.mux);
|
||||
e.bytes += packet.size;
|
||||
result = av_interleaved_write_frame(e.format, packet); // takes the packet's data
|
||||
unlock(*e.mux);
|
||||
if result < 0 return false, av_error("FFmpeg cannot write to the MP4", result);
|
||||
}
|
||||
return true, "";
|
||||
}
|
||||
|
||||
encoder_open_audio :: (e: *Encoder) -> ok: bool, error: string {
|
||||
codec := avcodec_find_encoder_by_name("aac");
|
||||
if !codec return false, "this FFmpeg has no AAC encoder";
|
||||
c := avcodec_alloc_context3(codec);
|
||||
if !c return false, "out of memory";
|
||||
c.sample_fmt = .FLTP;
|
||||
c.sample_rate = AUDIO_RATE;
|
||||
av_channel_layout_default(*c.ch_layout, AUDIO_CHANNELS);
|
||||
c.bit_rate = RECORD_AUDIO_BIT_RATE;
|
||||
c.time_base = .{1, AUDIO_RATE};
|
||||
if e.format.oformat.flags & AVFMT_GLOBALHEADER c.flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
|
||||
result := avcodec_open2(c, codec, null);
|
||||
if result < 0 { avcodec_free_context(*c); return false, av_error("the AAC encoder cannot open", result); }
|
||||
e.audio = c;
|
||||
e.audio_frame_size = ifx c.frame_size > 0 then c.frame_size else 1024;
|
||||
|
||||
e.audio_stream = avformat_new_stream(e.format, null);
|
||||
if !e.audio_stream return false, "FFmpeg cannot add a sound stream";
|
||||
avcodec_parameters_from_context(e.audio_stream.codecpar, c);
|
||||
e.audio_stream.time_base = c.time_base;
|
||||
|
||||
e.audio_packet = av_packet_alloc();
|
||||
e.audio_frame = av_frame_alloc();
|
||||
if !e.audio_packet || !e.audio_frame return false, "FFmpeg is out of memory";
|
||||
e.audio_frame.format = cast(s32) AVSampleFormat.FLTP;
|
||||
e.audio_frame.nb_samples = e.audio_frame_size;
|
||||
e.audio_frame.sample_rate = AUDIO_RATE;
|
||||
av_channel_layout_default(*e.audio_frame.ch_layout, AUDIO_CHANNELS);
|
||||
result = av_frame_get_buffer(e.audio_frame, 0);
|
||||
if result < 0 return false, av_error("FFmpeg cannot make a sound frame", result);
|
||||
return true, "";
|
||||
}
|
||||
|
||||
encoder_free :: (e: *Encoder) {
|
||||
if e.audio_frame av_frame_free(*e.audio_frame);
|
||||
if e.audio_packet av_packet_free(*e.audio_packet);
|
||||
if e.audio avcodec_free_context(*e.audio);
|
||||
e.audio_stream = null;
|
||||
if e.mux_ready { destroy(*e.mux); e.mux_ready = false; }
|
||||
if e.sws sws_freeContext(e.sws);
|
||||
if e.frame av_frame_free(*e.frame);
|
||||
if e.packet av_packet_free(*e.packet);
|
||||
@ -205,6 +298,8 @@ encoder_free :: (e: *Encoder) {
|
||||
e.sws, e.format, e.stream = null, null, null;
|
||||
}
|
||||
|
||||
#scope_module
|
||||
|
||||
// What FFmpeg says an error code is, after what failed (temp).
|
||||
av_error :: (what: string, code: s32) -> string {
|
||||
buffer: [256] u8;
|
||||
|
||||
@ -13,6 +13,13 @@
|
||||
// one a second while nothing does, so players can seek. Each carries the time it was taken, less the
|
||||
// time spent paused (record_pause), so a pause leaves no gap in the video.
|
||||
//
|
||||
// Sound (system_audio, microphone) has a thread of its own (audio_source.jai's WASAPI): each packet is
|
||||
// put on the same timeline as the frames, by when it was heard, less the same pauses, into one mix,
|
||||
// and an AAC frame of it is encoded once every source has had its say up to there. A source that sends
|
||||
// nothing (the computer playing nothing) is silence. Either source can be turned on and off while
|
||||
// recording (record_set_audio); with audio_track, the MP4 has a sound track even when no source is on
|
||||
// at the start, so one can be.
|
||||
//
|
||||
// recorder: Recorder;
|
||||
// ok, error, fall_back := record_start(*recorder, .{path = "out.mp4", screen = 0});
|
||||
// ...
|
||||
@ -66,6 +73,9 @@ Record_Options :: struct {
|
||||
capture: Record_Capture;
|
||||
legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe
|
||||
testing: Record_Testing;
|
||||
system_audio: bool; // what the computer plays (the default output device)
|
||||
microphone: bool; // the default input device; both at the start: record_set_audio turns them on and off
|
||||
audio_track: bool; // a sound track whatever is on at the start, so a source can be turned on later
|
||||
}
|
||||
|
||||
Record_Stats :: struct {
|
||||
@ -81,6 +91,9 @@ Record_Stats :: struct {
|
||||
dropped: s64; // the encoder was too far behind
|
||||
lost: s64; // times the duplication was lost and made again
|
||||
paused_seconds: float64; // left out of the video
|
||||
audio: string; // the sound recorded: "Speakers (Realtek(R) Audio); Microphone (USB Audio)", or ""
|
||||
audio_frames: s64; // AAC frames encoded
|
||||
audio_errors: string; // sources that could not be opened, or were lost
|
||||
copy_ticks, readback_ticks, encode_ticks: s64; // totals, in QueryPerformanceCounter ticks
|
||||
bytes: s64; // encoded
|
||||
seconds: float64; // from start to stop
|
||||
@ -99,7 +112,17 @@ Recorder :: struct {
|
||||
start_ticks: s64;
|
||||
|
||||
capture_thread, encoder_thread: Thread;
|
||||
capture_started, encoder_started: bool;
|
||||
capture_started, encoder_started, audio_started: bool;
|
||||
audio_thread: Thread;
|
||||
audio_opened: Semaphore; // the audio thread has opened its sources
|
||||
audio_go: Semaphore; // the encoder is open: the audio thread may write
|
||||
audio_sources: [2] Audio_Source;
|
||||
audio_open: bool; // any source opened, or audio_track: the MP4 has sound
|
||||
audio_on: [2] bool; // each source is wanted (by Audio_Source_Kind): record_set_audio; cleared when it cannot be opened
|
||||
audio_failed: [2] bool; // it could not be opened after being turned on: record_audio_failed says once
|
||||
zero_ticks: s64; // the recording's time 0, for frames and sound alike
|
||||
paused_ticks: s64; // time paused so far, for the capture and audio threads; set by record_pause
|
||||
pause_began: s64;
|
||||
running: bool;
|
||||
stopping: bool; // set by record_stop; the capture thread ends, then the encoder
|
||||
paused: bool; // set by record_pause; the capture thread takes nothing while it is
|
||||
@ -175,6 +198,7 @@ record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: stri
|
||||
QueryPerformanceFrequency(*frequency);
|
||||
r.frequency = frequency.QuadPart;
|
||||
r.start_ticks = ticks_now();
|
||||
r.zero_ticks = r.start_ticks;
|
||||
|
||||
// The source is opened on the capture thread, which uses it: WinRT's objects belong to the
|
||||
// apartment they are made in.
|
||||
@ -188,7 +212,19 @@ record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: stri
|
||||
return false, error, fall_back;
|
||||
}
|
||||
|
||||
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder);
|
||||
// The sound's sources are opened on their own thread (COM, for WASAPI), before the encoder, which
|
||||
// has a sound stream only when one opened.
|
||||
r.audio_on = .[r.options.system_audio, r.options.microphone];
|
||||
if r.options.system_audio || r.options.microphone || r.options.audio_track {
|
||||
init(*r.audio_opened);
|
||||
init(*r.audio_go);
|
||||
thread_init(*r.audio_thread, record_audio_proc, name = "record audio");
|
||||
r.audio_started = true;
|
||||
thread_start(*r.audio_thread, r);
|
||||
wait_for(*r.audio_opened);
|
||||
}
|
||||
|
||||
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder, audio = r.audio_open);
|
||||
if tried log("[record] Encoders that did not open: %", tried);
|
||||
if !ok {
|
||||
error = copy_string(error,, temp);
|
||||
@ -196,6 +232,7 @@ record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: stri
|
||||
return false, error, false;
|
||||
}
|
||||
r.stats.encoder = copy_string(r.encoder.name);
|
||||
if r.audio_started signal(*r.audio_go);
|
||||
|
||||
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
|
||||
r.encoder_started = true;
|
||||
@ -211,6 +248,7 @@ record_stop :: (r: *Recorder) -> ok: bool, error: string {
|
||||
atomic_swap(*r.stopping, true);
|
||||
thread_is_done(*r.capture_thread, -1);
|
||||
thread_is_done(*r.encoder_thread, -1);
|
||||
if r.audio_started thread_is_done(*r.audio_thread, -1);
|
||||
r.running = false;
|
||||
|
||||
closed, close_error := encoder_close(*r.encoder);
|
||||
@ -224,9 +262,44 @@ record_stop :: (r: *Recorder) -> ok: bool, error: string {
|
||||
return true, "";
|
||||
}
|
||||
|
||||
// Pauses or resumes. While paused nothing is taken, and the time is left out of the video.
|
||||
// Pauses or resumes. While paused nothing is taken, and the time is left out of the video, sound and
|
||||
// all: the time paused is kept here, once, for the capture and audio threads both.
|
||||
record_pause :: (r: *Recorder, pause: bool) {
|
||||
if r.running atomic_swap(*r.paused, pause);
|
||||
if !r.running || atomic_read(*r.paused) == pause return;
|
||||
now := ticks_now();
|
||||
if pause {
|
||||
atomic_write(*r.pause_began, now);
|
||||
atomic_swap(*r.paused, true);
|
||||
} else {
|
||||
atomic_add(*r.paused_ticks, now - atomic_read(*r.pause_began)); // before paused clears, so no thread sees it unpaused with the old total
|
||||
atomic_swap(*r.paused, false);
|
||||
}
|
||||
}
|
||||
|
||||
// Turns a source of sound on or off while recording. It is opened only while on, so Windows shows the
|
||||
// microphone in use only then. False when the recording has no sound track to put it in.
|
||||
record_set_audio :: (r: *Recorder, kind: Audio_Source_Kind, on: bool) -> bool {
|
||||
if !r.running || !r.audio_open return false;
|
||||
atomic_swap(*r.audio_on[kind], on);
|
||||
return true;
|
||||
}
|
||||
|
||||
record_audio_on :: (r: *Recorder, kind: Audio_Source_Kind) -> bool {
|
||||
return r.running && atomic_read(*r.audio_on[kind]);
|
||||
}
|
||||
|
||||
// Whether a source could not be opened when turned on (it is off again then), and why; once.
|
||||
record_audio_failed :: (r: *Recorder, kind: Audio_Source_Kind) -> failed: bool, error: string {
|
||||
if !r.running || !atomic_read(*r.audio_failed[kind]) return false, "";
|
||||
atomic_swap(*r.audio_failed[kind], false);
|
||||
lock(*r.mutex);
|
||||
defer unlock(*r.mutex);
|
||||
return true, copy_string(r.stats.audio_errors,, temp);
|
||||
}
|
||||
|
||||
// Ticks on the recording's timeline: time 0 at its start, less the time paused.
|
||||
record_timeline :: (r: *Recorder, ticks: s64) -> s64 {
|
||||
return ticks - r.zero_ticks - atomic_read(*r.paused_ticks);
|
||||
}
|
||||
|
||||
record_paused :: (r: *Recorder) -> bool {
|
||||
@ -265,6 +338,14 @@ record_free :: (r: *Recorder) {
|
||||
atomic_swap(*r.stopping, true);
|
||||
if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; }
|
||||
if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; }
|
||||
if r.audio_started {
|
||||
signal(*r.audio_go); // in case it waits for an encoder that did not open
|
||||
thread_is_done(*r.audio_thread, -1);
|
||||
thread_deinit(*r.audio_thread);
|
||||
destroy(*r.audio_opened);
|
||||
destroy(*r.audio_go);
|
||||
r.audio_started = false;
|
||||
}
|
||||
if r.encoder.format encoder_close(*r.encoder);
|
||||
for * r.frames array_free(it.pixels);
|
||||
array_free(r.last_frame);
|
||||
@ -331,7 +412,7 @@ record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
have_image, changed := false, false;
|
||||
staged: [RECORD_STAGING] Staged;
|
||||
emits: s64;
|
||||
paused_ticks, pause_start: s64; // time paused so far, left out of the frames' times; when this pause began
|
||||
was_paused := false;
|
||||
last_lost := ticks_now(); // for Record_Testing.LOSE_DUPLICATION
|
||||
|
||||
while !atomic_read(*r.stopping) {
|
||||
@ -381,15 +462,15 @@ record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
// screen as it is now is sent at once.
|
||||
now = ticks_now();
|
||||
if atomic_read(*r.paused) {
|
||||
if !pause_start pause_start = now;
|
||||
lock(*r.mutex); r.stats.paused_seconds = cast(float64) (paused_ticks + now - pause_start) / r.frequency; unlock(*r.mutex);
|
||||
was_paused = true;
|
||||
lock(*r.mutex); r.stats.paused_seconds = cast(float64) (atomic_read(*r.paused_ticks) + now - atomic_read(*r.pause_began)) / r.frequency; unlock(*r.mutex);
|
||||
continue;
|
||||
}
|
||||
if pause_start {
|
||||
paused_ticks += now - pause_start;
|
||||
pause_start = 0;
|
||||
if was_paused {
|
||||
was_paused = false;
|
||||
changed = true;
|
||||
turn = now;
|
||||
lock(*r.mutex); r.stats.paused_seconds = cast(float64) atomic_read(*r.paused_ticks) / r.frequency; unlock(*r.mutex);
|
||||
}
|
||||
if !have_image continue;
|
||||
repeat := !changed && now - last_emit >= r.frequency;
|
||||
@ -400,7 +481,7 @@ record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
slot := emits % RECORD_STAGING;
|
||||
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
|
||||
ID3D11DeviceContext_CopyResource(gpu.device_context, gpu.staging[slot], gpu.latest);
|
||||
staged[slot] = .{pending = true, ticks = now - paused_ticks};
|
||||
staged[slot] = .{pending = true, ticks = record_timeline(r, now)};
|
||||
if !r.using_wgc staged[slot].pointer, staged[slot].pointer_visible = r.dxgi.pointer_position, r.dxgi.pointer_visible;
|
||||
emits += 1;
|
||||
last_emit = now;
|
||||
@ -465,7 +546,6 @@ record_read_back :: (r: *Recorder, gpu: *Capture_Gpu, slot: s64, staged: Staged)
|
||||
|
||||
record_encoder_proc :: (thread: *Thread) -> s64 {
|
||||
r := cast(*Recorder) thread.data;
|
||||
first: s64 = -1;
|
||||
while true {
|
||||
wait_for(*r.ready, 100);
|
||||
lock(*r.mutex);
|
||||
@ -482,8 +562,7 @@ record_encoder_proc :: (thread: *Thread) -> s64 {
|
||||
}
|
||||
|
||||
frame := *r.frames[index];
|
||||
if first < 0 first = frame.ticks;
|
||||
pts := (frame.ticks - first) * 90000 / r.frequency;
|
||||
pts := max(frame.ticks, 0) * 90000 / r.frequency; // frame.ticks is on the timeline already
|
||||
t := ticks_now();
|
||||
ok, error := encoder_write(*r.encoder, frame.pixels.data, pts);
|
||||
|
||||
@ -510,6 +589,171 @@ record_encoder_proc :: (thread: *Thread) -> s64 {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// The audio thread: opens the sources asked for, waits for the encoder, then every 10 ms takes their
|
||||
// packets into the mix and encodes what every source has covered.
|
||||
record_audio_proc :: (thread: *Thread) -> s64 {
|
||||
r := cast(*Recorder) thread.data;
|
||||
audio_thread_start();
|
||||
defer audio_thread_end();
|
||||
|
||||
kinds: [2] Audio_Source_Kind = .[.SYSTEM, .MICROPHONE];
|
||||
wanted: [2] bool = .[r.options.system_audio, r.options.microphone];
|
||||
names, errors: [..] string; // every source heard, and every failure, for the stats
|
||||
names.allocator, errors.allocator = context.default_allocator, context.default_allocator;
|
||||
defer { for names free(it); array_free(names); for errors free(it); array_free(errors); }
|
||||
note :: (list: *[..] string, text: string, field: *string) #expand {
|
||||
for list.* if it == text return;
|
||||
array_add(list, copy_string(text));
|
||||
lock(*`r.mutex);
|
||||
free(field.*);
|
||||
field.* = join(..list.*, "; ");
|
||||
unlock(*`r.mutex);
|
||||
}
|
||||
for 0..1 {
|
||||
if !wanted[it] continue;
|
||||
s := *r.audio_sources[it];
|
||||
ok, error := audio_open(s, kinds[it]);
|
||||
what := ifx kinds[it] == .SYSTEM then "system sound" else "microphone";
|
||||
if ok { r.audio_open = true; note(*names, s.name, *r.stats.audio); }
|
||||
else { audio_close(s); note(*errors, tprint("%: %", what, error), *r.stats.audio_errors); log_error("[record] No %: %", what, error); }
|
||||
}
|
||||
for r.audio_sources if !it.open atomic_swap(*r.audio_on[it_index], false);
|
||||
if r.options.audio_track r.audio_open = true;
|
||||
signal(*r.audio_opened);
|
||||
wait_for(*r.audio_go);
|
||||
if !r.audio_open || !r.encoder.audio || atomic_read(*r.stopping) {
|
||||
for * r.audio_sources audio_free(it);
|
||||
return 0;
|
||||
}
|
||||
|
||||
mix: Audio_Mix;
|
||||
mix.frame_size = r.encoder.audio_frame_size;
|
||||
defer array_free(mix.samples);
|
||||
while true {
|
||||
stopping := atomic_read(*r.stopping);
|
||||
// Each source, turned on or off from the toolbar: opened only while on.
|
||||
for * source, index: r.audio_sources {
|
||||
on := atomic_read(*r.audio_on[index]) && !stopping;
|
||||
what := ifx kinds[index] == .SYSTEM then "System sound" else "Microphone";
|
||||
if on && !source.open && !wanted[index] {
|
||||
ok, error := audio_open(source, kinds[index]);
|
||||
if ok {
|
||||
log("[record] % on: %.", what, source.name);
|
||||
note(*names, source.name, *r.stats.audio);
|
||||
mix.started[index] = false;
|
||||
} else {
|
||||
audio_close(source);
|
||||
log_error("[record] % could not be opened: %", what, error);
|
||||
note(*errors, tprint("%: %", to_lower_copy(what,, temp), error), *r.stats.audio_errors);
|
||||
atomic_swap(*r.audio_on[index], false);
|
||||
atomic_swap(*r.audio_failed[index], true);
|
||||
}
|
||||
} else if !on && source.open {
|
||||
log("[record] % off.", what);
|
||||
audio_close(source);
|
||||
source.lost_at = 0;
|
||||
}
|
||||
wanted[index] = source.open || (wanted[index] && on);
|
||||
}
|
||||
|
||||
for * source, index: r.audio_sources {
|
||||
if !wanted[index] continue;
|
||||
if !source.open { // lost: opened again a second on, from the default device then
|
||||
if source.lost_at && ticks_now() - source.lost_at >= r.frequency && !stopping {
|
||||
if audio_open(source, kinds[index]) log("[record] % is back: %.", ifx kinds[index] == .SYSTEM then "System sound" else "The microphone", source.name);
|
||||
else { audio_close(source); source.lost_at = ticks_now(); }
|
||||
}
|
||||
continue;
|
||||
}
|
||||
packets, lost := audio_take(source, r.frequency);
|
||||
for packets if !atomic_read(*r.paused) audio_mix_add(*mix, r, index, it);
|
||||
if lost {
|
||||
log_error("[record] % was lost (unplugged, or the default device changed); trying again.", source.name);
|
||||
audio_close(source);
|
||||
source.lost_at = ticks_now();
|
||||
}
|
||||
}
|
||||
ok, error := audio_mix_encode(*mix, r, final = stopping);
|
||||
if !ok { record_fail(r, error); break; }
|
||||
if stopping break;
|
||||
reset_temporary_storage();
|
||||
sleep_milliseconds(10);
|
||||
}
|
||||
for * r.audio_sources audio_free(it);
|
||||
return 0;
|
||||
}
|
||||
|
||||
// The sound so far, from its base: one AAC frame's worth is encoded at a time.
|
||||
Audio_Mix :: struct {
|
||||
samples: [..] float; // interleaved stereo, from base on
|
||||
base: s64; // the frame (in samples, on the timeline) samples[0] is
|
||||
next: [2] s64; // each source's next frame, so packets that follow on are not moved by jitter
|
||||
started: [2] bool;
|
||||
frame_size: s32;
|
||||
}
|
||||
|
||||
AUDIO_SLACK :: AUDIO_RATE / 5; // 200 ms: how late a source's packets may come and still be in the mix
|
||||
AUDIO_FOLLOW_ON :: AUDIO_RATE / 33; // 30 ms: a packet that close to where its source left off follows on
|
||||
|
||||
// A packet into the mix at its place on the timeline, added to what the other source put there.
|
||||
audio_mix_add :: (mix: *Audio_Mix, r: *Recorder, source: s64, packet: Audio_Packet) {
|
||||
at := record_timeline(r, packet.ticks) * AUDIO_RATE / r.frequency;
|
||||
if mix.started[source] && abs(at - mix.next[source]) <= AUDIO_FOLLOW_ON at = mix.next[source];
|
||||
mix.started[source] = true;
|
||||
mix.next[source] = at + packet.frames;
|
||||
end := (at + packet.frames - mix.base) * AUDIO_CHANNELS;
|
||||
if end > mix.samples.count {
|
||||
old := mix.samples.count;
|
||||
array_resize(*mix.samples, end, initialize = false);
|
||||
for old..end - 1 mix.samples[it] = 0;
|
||||
}
|
||||
if packet.silent || !packet.samples.count return;
|
||||
for frame: max(at, mix.base)..at + packet.frames - 1 {
|
||||
into := (frame - mix.base) * AUDIO_CHANNELS;
|
||||
from := (frame - at) * AUDIO_CHANNELS;
|
||||
mix.samples[into] += packet.samples[from];
|
||||
mix.samples[into + 1] += packet.samples[from + 1];
|
||||
}
|
||||
}
|
||||
|
||||
// Encodes the frames every open source has covered: up to where each has sent, or, for one that has
|
||||
// sent nothing lately (silence), to AUDIO_SLACK ago. final: everything sent.
|
||||
audio_mix_encode :: (mix: *Audio_Mix, r: *Recorder, final: bool) -> ok: bool, error: string {
|
||||
// Paused, the timeline stands still where the pause began.
|
||||
now := ifx atomic_read(*r.paused) then atomic_read(*r.pause_began) else ticks_now();
|
||||
now_frame := record_timeline(r, now) * AUDIO_RATE / r.frequency;
|
||||
ready := now_frame - AUDIO_SLACK;
|
||||
for source, index: r.audio_sources {
|
||||
if !source.open continue;
|
||||
covered := max(mix.next[index], now_frame - AUDIO_SLACK);
|
||||
ready = min(ready, covered);
|
||||
}
|
||||
if final { ready = mix.base; for mix.next ready = max(ready, it); }
|
||||
|
||||
size := mix.frame_size;
|
||||
frame := NewArray(size * AUDIO_CHANNELS, float, initialized = false,, temp);
|
||||
done: s64;
|
||||
while mix.base + size <= ready {
|
||||
for i: 0..frame.count - 1 {
|
||||
at := done * AUDIO_CHANNELS + i;
|
||||
value := ifx at < mix.samples.count then mix.samples[at] else 0.0;
|
||||
frame[i] = clamp(value, -1.0, 1.0);
|
||||
}
|
||||
ok, error := encoder_write_audio(*r.encoder, frame.data, mix.base);
|
||||
if !ok return false, error;
|
||||
mix.base += size;
|
||||
done += size;
|
||||
lock(*r.mutex); r.stats.audio_frames += 1; unlock(*r.mutex);
|
||||
}
|
||||
// What was encoded leaves the front.
|
||||
gone := min(done * AUDIO_CHANNELS, mix.samples.count);
|
||||
if gone {
|
||||
memcpy(mix.samples.data, mix.samples.data + gone, (mix.samples.count - gone) * size_of(float));
|
||||
mix.samples.count -= gone;
|
||||
}
|
||||
return true, "";
|
||||
}
|
||||
|
||||
// Ends the recording from one of its threads; the first reason is kept.
|
||||
record_fail :: (r: *Recorder, error: string) {
|
||||
log_error("[record] %", error);
|
||||
@ -529,11 +773,14 @@ rect_inside :: (inner: RECT, outer: RECT) -> bool {
|
||||
#load "dxgi_source.jai";
|
||||
#load "wgc_source.jai";
|
||||
#load "encoder.jai";
|
||||
#load "remux.jai";
|
||||
#load "audio_source.jai";
|
||||
#load "cursor.jai";
|
||||
|
||||
#import "Basic";
|
||||
#import "String";
|
||||
#import "Atomics";
|
||||
#import "Math";
|
||||
#import "Thread";
|
||||
#import "Windows";
|
||||
#import "Windows_Utf8";
|
||||
|
||||
65
modules/Screen_Record/remux.jai
Normal file
65
modules/Screen_Record/remux.jai
Normal file
@ -0,0 +1,65 @@
|
||||
// A recording made into a regular MP4. While recording, the MP4 is written fragmented (encoder.jai), so
|
||||
// a crash still leaves one that plays; but a fragmented MP4's header says nothing of its length or
|
||||
// where its frames are, and one cut short by a crash never gets the index written at the end of a
|
||||
// clean stop, so players show its time and seek in it badly. record_remux copies its packets, as they
|
||||
// are, into a regular MP4: one header, with the length and the index of every frame, at the front
|
||||
// (faststart), so it plays, seeks and streams as any MP4 does. Nothing is encoded again.
|
||||
|
||||
// Copies source's video and audio packets into a regular MP4 at destination. packets: how many.
|
||||
record_remux :: (source: string, destination: string) -> ok: bool, error: string, packets: s64 {
|
||||
av_log_set_level(AV_LOG_ERROR);
|
||||
input: *AVFormatContext;
|
||||
result := avformat_open_input(*input, temp_c_string(source), null, null);
|
||||
if result < 0 return false, av_error("FFmpeg cannot open the recording", result), 0;
|
||||
defer avformat_close_input(*input);
|
||||
result = avformat_find_stream_info(input, null);
|
||||
if result < 0 return false, av_error("FFmpeg cannot read the recording's streams", result), 0;
|
||||
|
||||
output: *AVFormatContext;
|
||||
result = avformat_alloc_output_context2(*output, null, "mp4", temp_c_string(destination));
|
||||
if result < 0 || !output return false, av_error("FFmpeg cannot make an MP4", result), 0;
|
||||
defer {
|
||||
if output.pb avio_closep(*output.pb);
|
||||
avformat_free_context(output);
|
||||
}
|
||||
|
||||
// Each input stream's output stream, or -1 for one left out (neither video nor audio).
|
||||
map := NewArray(input.nb_streams, s32,, temp);
|
||||
for 0..input.nb_streams - 1 {
|
||||
in := input.streams[it];
|
||||
map[it] = -1;
|
||||
if in.codecpar.codec_type != .VIDEO && in.codecpar.codec_type != .AUDIO continue;
|
||||
out := avformat_new_stream(output, null);
|
||||
if !out return false, "FFmpeg cannot add a stream", 0;
|
||||
avcodec_parameters_copy(out.codecpar, in.codecpar);
|
||||
out.codecpar.codec_tag = 0; // the MP4 muxer picks its own
|
||||
out.time_base = in.time_base;
|
||||
map[it] = out.index;
|
||||
}
|
||||
|
||||
result = avio_open(*output.pb, temp_c_string(destination), AVIO_FLAG_WRITE);
|
||||
if result < 0 return false, av_error(tprint("FFmpeg cannot write %", destination), result), 0;
|
||||
options: *AVDictionary;
|
||||
av_dict_set(*options, "movflags", "+faststart", 0); // the header before the frames: a second pass at the end
|
||||
result = avformat_write_header(output, *options);
|
||||
av_dict_free(*options);
|
||||
if result < 0 return false, av_error("FFmpeg cannot start the MP4", result), 0;
|
||||
|
||||
packet := av_packet_alloc();
|
||||
defer av_packet_free(*packet);
|
||||
packets: s64;
|
||||
while av_read_frame(input, packet) >= 0 { // to the end, or to where a crash cut it
|
||||
index := map[packet.stream_index];
|
||||
if index < 0 { av_packet_unref(packet); continue; }
|
||||
av_packet_rescale_ts(packet, input.streams[packet.stream_index].time_base, output.streams[index].time_base);
|
||||
packet.stream_index = index;
|
||||
packet.pos = -1;
|
||||
result = av_interleaved_write_frame(output, packet); // takes the packet's data
|
||||
if result < 0 return false, av_error("FFmpeg cannot write to the MP4", result), packets;
|
||||
packets += 1;
|
||||
}
|
||||
if !packets return false, "the recording holds no frames", 0;
|
||||
result = av_write_trailer(output);
|
||||
if result < 0 return false, av_error("FFmpeg cannot finish the MP4", result), packets;
|
||||
return true, "", packets;
|
||||
}
|
||||
@ -308,24 +308,6 @@ IID_DIRECT3D_DEVICE := #run guid("A37624AB-8D5F-4650-9D3E-9EAE3D9BC670");
|
||||
IID_DXGI_INTERFACE_ACCESS := #run guid("A9B3D012-3DF2-4EE3-B8D1-8695F457D3C1");
|
||||
IID_CLOSABLE := #run guid("30D5A829-7FA4-4026-83BB-D75BAE4EA99E");
|
||||
|
||||
guid :: (text: string) -> GUID {
|
||||
hex :: (s: string) -> u64 {
|
||||
value: u64;
|
||||
for 0..s.count - 1 {
|
||||
c := s[it];
|
||||
digit: u64 = ifx c >= #char "a" then cast(u64) (c - #char "a" + 10) else ifx c >= #char "A" then cast(u64) (c - #char "A" + 10) else cast(u64) (c - #char "0");
|
||||
value = value * 16 + digit;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
g: GUID;
|
||||
g.Data1 = cast(u32) hex(slice(text, 0, 8));
|
||||
g.Data2 = cast(u16) hex(slice(text, 9, 4));
|
||||
g.Data3 = cast(u16) hex(slice(text, 14, 4));
|
||||
for 0..1 g.Data4[it] = cast(u8) hex(slice(text, 19 + it * 2, 2));
|
||||
for 0..5 g.Data4[it + 2] = cast(u8) hex(slice(text, 24 + it * 2, 2));
|
||||
return g;
|
||||
}
|
||||
|
||||
// IUnknown, then IInspectable's three: every WinRT interface starts so.
|
||||
Inspectable_Methods :: struct {
|
||||
|
||||
@ -23,6 +23,7 @@ Screen_Capture_Job :: struct {
|
||||
source: string; // how the screen was taken: Record_Stats.source
|
||||
dropped: s64; // frames the encoder fell too far behind for
|
||||
paused_seconds: float64; // left out of it
|
||||
audio: string; // the sound's sources, "Speakers (...); Microphone (...)", or ""
|
||||
|
||||
thread: Thread;
|
||||
busy: bool; // cleared by the thread as it ends
|
||||
@ -142,6 +143,7 @@ screen_capture_job_free :: (job: *Screen_Capture_Job) {
|
||||
free(job.recording);
|
||||
free(job.encoder);
|
||||
free(job.source);
|
||||
free(job.audio);
|
||||
free(job);
|
||||
}
|
||||
|
||||
|
||||
@ -9,8 +9,10 @@
|
||||
//
|
||||
// While it records, a red border is drawn just outside the region and a toolbar beside it gives the
|
||||
// time, Pause, Stop and Discard (app/screen_record_win32.jai); both are kept out of the recording.
|
||||
// Time paused is left out of the video. The frame rate, the cursor and how the screen is taken are
|
||||
// [video] in the settings file.
|
||||
// Time paused is left out of the video. With sound: what the computer plays and the microphone, as
|
||||
// [video] in the settings file says, as are the frame rate, the cursor and how the screen is taken.
|
||||
// The toolbar's System sound and Mic buttons turn each on and off while recording, and are kept in
|
||||
// [video] for the next: every recording has a sound track for them, silent while nothing is on.
|
||||
//
|
||||
// A recording that stops by itself (the screen went away, the encoder failed) keeps what it has, and
|
||||
// so does quitting while recording. One left in the temporary folder by a crash is offered at the next
|
||||
@ -50,7 +52,25 @@ screen_record_frame :: () {
|
||||
screen_record_stop();
|
||||
return;
|
||||
}
|
||||
if record_ui_take_ask() == {
|
||||
for kind: Audio_Source_Kind.[.SYSTEM, .MICROPHONE] {
|
||||
failed, why := record_audio_failed(*screen_record.recorder, kind);
|
||||
if !failed continue;
|
||||
toast_show(.Error, "% could not be turned on: %", screen_record_audio_name(kind), why);
|
||||
screen_record_remember_audio(kind, false);
|
||||
record_ui_refresh();
|
||||
}
|
||||
ask := record_ui_take_ask();
|
||||
if ask == .SYSTEM_SOUND || ask == .MICROPHONE {
|
||||
kind: Audio_Source_Kind = ifx ask == .SYSTEM_SOUND then .SYSTEM else .MICROPHONE;
|
||||
on := !record_audio_on(*screen_record.recorder, kind);
|
||||
if record_set_audio(*screen_record.recorder, kind, on) {
|
||||
log("[record] % turned %.", screen_record_audio_name(kind), ifx on then "on" else "off");
|
||||
toast_show(.Success, "% %", screen_record_audio_name(kind), ifx on then "on" else "off");
|
||||
screen_record_remember_audio(kind, on);
|
||||
}
|
||||
record_ui_refresh();
|
||||
}
|
||||
if ask == {
|
||||
case .PAUSE;
|
||||
pause := !record_paused(*screen_record.recorder);
|
||||
record_pause(*screen_record.recorder, pause);
|
||||
@ -204,6 +224,9 @@ screen_record_start :: (immediate := false) {
|
||||
options.region = .{shot.left, shot.top, shot.left + shot.width, shot.top + shot.height};
|
||||
options.fps = cast(s32) app_user_settings.video_fps;
|
||||
options.cursor = app_user_settings.video_cursor;
|
||||
options.system_audio = app_user_settings.video_system_audio;
|
||||
options.microphone = app_user_settings.video_microphone;
|
||||
options.audio_track = true; // so the toolbar can turn a source on mid-recording
|
||||
if app_user_settings.video_capture == {
|
||||
case "dxgi"; options.capture = .DXGI;
|
||||
case "wgc"; options.capture = .WGC;
|
||||
@ -224,10 +247,28 @@ screen_record_start :: (immediate := false) {
|
||||
if stats.fell_back_from log("[record] Desktop Duplication could not run: %", stats.fell_back_from);
|
||||
log("[record] Recording % x % at %, % (%) of \"%\" (%) with % and %, to %.", stats.width, stats.height, shot.left, shot.top,
|
||||
ifx shot.whole_screen then "the whole screen" else "a region", shot.window_title, shot.app_name, stats.source, stats.encoder, path);
|
||||
toast_show(.Success, "Recording % x %: % stops", stats.width, stats.height, screen_record_hotkey_text());
|
||||
if stats.audio log("[record] Sound: %.", stats.audio);
|
||||
if stats.audio_errors {
|
||||
log_error("[record] Sound not recorded: %.", stats.audio_errors);
|
||||
toast_show(.Error, "Recording without some sound: %", stats.audio_errors);
|
||||
} else {
|
||||
toast_show(.Success, "Recording % x %: % stops", stats.width, stats.height, screen_record_hotkey_text());
|
||||
}
|
||||
record_ui_open(options.region, shot.whole_screen, stats.width, stats.height);
|
||||
}
|
||||
|
||||
// A sound button's state, into [video] for the next recording.
|
||||
screen_record_remember_audio :: (kind: Audio_Source_Kind, on: bool) {
|
||||
setting := ifx kind == .SYSTEM then *app_user_settings.video_system_audio else *app_user_settings.video_microphone;
|
||||
if setting.* == on return;
|
||||
setting.* = on;
|
||||
app_save_user_settings_file();
|
||||
}
|
||||
|
||||
screen_record_audio_name :: (kind: Audio_Source_Kind) -> string {
|
||||
return ifx kind == .SYSTEM then "System sound" else "The microphone";
|
||||
}
|
||||
|
||||
// Stops, and deletes what was recorded (the toolbar's Discard, which asked first).
|
||||
screen_record_discard :: () {
|
||||
if !screen_record.recording return;
|
||||
@ -278,6 +319,7 @@ screen_record_stop :: (shutdown := false) {
|
||||
job.source = copy_string(stats.source,, context.default_allocator);
|
||||
job.dropped = stats.dropped;
|
||||
job.paused_seconds = stats.paused_seconds;
|
||||
job.audio = copy_string(stats.audio,, context.default_allocator);
|
||||
screen_record.shot = .{};
|
||||
screen_record.date, screen_record.path = "", "";
|
||||
screen_capture_job_start(job);
|
||||
@ -285,21 +327,34 @@ screen_record_stop :: (shutdown := false) {
|
||||
|
||||
// The recording's .mcap in the library, from the job's MP4, which is deleted once it is in: its path,
|
||||
// or why it could not be (temp). On the job's thread.
|
||||
//
|
||||
// The MP4 was written fragmented, so a crash leaves one that plays; it is made a regular MP4 first
|
||||
// (record_remux), with its length and index in its header, as players need to show its time and seek
|
||||
// in it. Should that fail, the MP4 goes in as it was.
|
||||
screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string {
|
||||
_, size, found := file_modtime_and_size(job.recording);
|
||||
if !found || size <= 0 return "", tprint("% is missing", job.recording);
|
||||
image, video, decoded, decode_error := video_poster(job.recording);
|
||||
_, recorded_size, found := file_modtime_and_size(job.recording);
|
||||
if !found || recorded_size <= 0 return "", tprint("% is missing", job.recording);
|
||||
mp4 := job.recording;
|
||||
remuxed := tprint("%.regular.mp4", path_strip_extension(job.recording));
|
||||
remux_ok, remux_error, packets := record_remux(job.recording, remuxed);
|
||||
if remux_ok mp4 = remuxed;
|
||||
else { log_error("[record] % stays fragmented: %", job.recording, remux_error); file_delete(remuxed); }
|
||||
defer if remux_ok file_delete(remuxed);
|
||||
_, size, _ := file_modtime_and_size(mp4);
|
||||
|
||||
image, video, decoded, decode_error := video_poster(mp4);
|
||||
if !decoded return "", tprint("the recording cannot be read back: %", decode_error);
|
||||
defer image_free(*image);
|
||||
preview, preview_width, preview_height, preview_ok, preview_error := image_preview_png(image, MCAP_PREVIEW_MAX_SIDE);
|
||||
if !preview_ok return "", preview_error;
|
||||
poster, _, _, poster_ok, poster_error := image_preview_png(image, max(image.width, image.height)); // at its own size
|
||||
if !poster_ok return "", poster_error;
|
||||
hash, hashed := import_hash_file(.{path = job.recording, offset = 0, size = size});
|
||||
if !hashed return "", tprint("% cannot be read", job.recording);
|
||||
hash, hashed := import_hash_file(.{path = mp4, offset = 0, size = size});
|
||||
if !hashed return "", tprint("% cannot be read", mp4);
|
||||
|
||||
shot := job.shot;
|
||||
if !shot.width { shot.width, shot.height = image.width, image.height; } // a crash's leftover: no more is known
|
||||
known := shot.width > 0; // a crash's leftover knows no region, window or encoder
|
||||
if !known { shot.width, shot.height = image.width, image.height; }
|
||||
doc := New(Capture_Document,, temp);
|
||||
capture_document_init(doc, temp);
|
||||
doc.comment = tprint("Recorded by % %.", APP_NAME, VERSION);
|
||||
@ -308,9 +363,11 @@ screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string
|
||||
import_add(doc, metadata, "CaptureDate", job.date);
|
||||
import_add(doc, metadata, "WindowName", import_text(shot.window_title));
|
||||
info := capture_add_file(doc, "capture.json");
|
||||
import_add(doc, info, "Region", ifx shot.whole_screen then "screen" else "selection");
|
||||
import_add_number(doc, info, "Left", xx shot.left);
|
||||
import_add_number(doc, info, "Top", xx shot.top);
|
||||
if known {
|
||||
import_add(doc, info, "Region", ifx shot.whole_screen then "screen" else "selection");
|
||||
import_add_number(doc, info, "Left", xx shot.left);
|
||||
import_add_number(doc, info, "Top", xx shot.top);
|
||||
}
|
||||
import_add_number(doc, info, "Width", xx shot.width);
|
||||
import_add_number(doc, info, "Height", xx shot.height);
|
||||
if shot.app_path import_add(doc, info, "AppPath", import_text(shot.app_path));
|
||||
@ -323,6 +380,8 @@ screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string
|
||||
if job.encoder import_add(doc, info, "Encoder", import_text(job.encoder));
|
||||
if job.source import_add(doc, info, "Capture", import_text(job.source));
|
||||
import_add_number(doc, info, "DroppedFrames", xx job.dropped);
|
||||
capture_object_add(doc, *info.root, "Audio", capture_boolean(video.has_audio));
|
||||
if job.audio import_add(doc, info, "AudioSources", import_text(job.audio));
|
||||
if job.paused_seconds > 0 import_add_number(doc, info, "Paused", import_milliseconds(job.paused_seconds));
|
||||
|
||||
name := import_out_name(job.date, hash);
|
||||
@ -337,7 +396,7 @@ screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string
|
||||
m.rendered = poster;
|
||||
path, folder_error := screen_capture_out_path(name);
|
||||
if folder_error return "", folder_error;
|
||||
written, write_error := mcap_write_streamed(path, m, .{path = job.recording, offset = 0, size = size});
|
||||
written, write_error := mcap_write_streamed(path, m, .{path = mp4, offset = 0, size = size});
|
||||
if !written return "", write_error;
|
||||
file_delete(job.recording);
|
||||
return path, "";
|
||||
|
||||
@ -1,5 +1,6 @@
|
||||
// While recording (app/screen_record.jai), on Windows: a thin red border just outside the region, and a
|
||||
// toolbar beside it with the time, the size, frames dropped, and Pause, Stop and Discard. Both are
|
||||
// toolbar beside it with the time, the size, frames dropped, and System sound, Mic, Pause, Stop and
|
||||
// Discard. Both are
|
||||
// plain GDI windows, as the overlay is, so they work while mcap is hidden in the tray, and both are
|
||||
// kept out of every capture (WDA_EXCLUDEFROMCAPTURE), the recording's own included. The border is
|
||||
// outside the region anyway, so it could never be in the video.
|
||||
@ -9,7 +10,7 @@
|
||||
// message, as Stop and Discard close the toolbar. Discard asks first, in the toolbar itself: a
|
||||
// message box would be in the recording.
|
||||
|
||||
Record_UI_Ask :: enum u8 { NONE; PAUSE; STOP; DISCARD; }
|
||||
Record_UI_Ask :: enum u8 { NONE; SYSTEM_SOUND; MICROPHONE; PAUSE; STOP; DISCARD; }
|
||||
|
||||
record_ui: struct {
|
||||
border, toolbar: HWND;
|
||||
@ -19,7 +20,7 @@ record_ui: struct {
|
||||
asked: Record_UI_Ask; // a button pressed, for screen_record_frame
|
||||
confirming: bool; // Discard pressed once: "Discard it? Yes No"
|
||||
hovered: s32 = -1; // the button under the mouse
|
||||
buttons: [3] Record_UI_Button;
|
||||
buttons: [5] Record_UI_Button;
|
||||
button_count: s32;
|
||||
started: Apollo_Time;
|
||||
size_text: string; // "1920 × 1080"; heap
|
||||
@ -170,10 +171,13 @@ record_toolbar_buttons :: () -> [] Record_UI_Button {
|
||||
record_ui.buttons[1] = .{label = "No", confirm = true};
|
||||
record_ui.button_count = 2;
|
||||
} else {
|
||||
record_ui.buttons[0] = .{label = ifx record_paused(*screen_record.recorder) then "Resume" else "Pause", ask = .PAUSE};
|
||||
record_ui.buttons[1] = .{label = "Stop", ask = .STOP};
|
||||
record_ui.buttons[2] = .{label = "Discard", confirm = true};
|
||||
record_ui.button_count = 3;
|
||||
recorder := *screen_record.recorder;
|
||||
record_ui.buttons[0] = .{label = ifx record_audio_on(recorder, .SYSTEM) then "System sound: on" else "System sound: off", ask = .SYSTEM_SOUND};
|
||||
record_ui.buttons[1] = .{label = ifx record_audio_on(recorder, .MICROPHONE) then "Mic: on" else "Mic: off", ask = .MICROPHONE};
|
||||
record_ui.buttons[2] = .{label = ifx record_paused(recorder) then "Resume" else "Pause", ask = .PAUSE};
|
||||
record_ui.buttons[3] = .{label = "Stop", ask = .STOP};
|
||||
record_ui.buttons[4] = .{label = "Discard", confirm = true};
|
||||
record_ui.button_count = 5;
|
||||
}
|
||||
return array_view(record_ui.buttons, 0, record_ui.button_count);
|
||||
}
|
||||
@ -196,7 +200,7 @@ record_toolbar_size :: () -> width: s32, height: s32 {
|
||||
padding, gap := record_ui_px(10), record_ui_px(8);
|
||||
status_width := text_width(dc, tprint("PAUSED 00:00:00 %", record_ui.size_text));
|
||||
dropped_width := text_width(dc, "999 dropped, screen lost 9 times");
|
||||
buttons_width := text_width(dc, "Resume") + text_width(dc, "Stop") + text_width(dc, "Yes, discard") + 3 * 2 * padding + 2 * gap;
|
||||
buttons_width := text_width(dc, "System sound: off") + text_width(dc, "Mic: off") + text_width(dc, "Resume") + text_width(dc, "Stop") + text_width(dc, "Yes, discard") + 5 * 2 * padding + 4 * gap;
|
||||
height := record_ui_px(34);
|
||||
width := padding + record_ui_px(12) + gap + status_width + gap + dropped_width + gap + buttons_width + padding;
|
||||
return width, height;
|
||||
@ -313,7 +317,9 @@ record_toolbar_paint :: (hwnd: HWND) {
|
||||
label_width := text_width(dc, it.label);
|
||||
r := RECT.{right - label_width - 2 * padding, (height - button_height) / 2, right, (height - button_height) / 2 + button_height};
|
||||
record_ui.buttons[it_index].rect = r;
|
||||
fill(r, ifx it_index == record_ui.hovered then cast(u32) RECORD_UI_HOVER else cast(u32) RECORD_UI_BUTTON);
|
||||
// A sound that is on, in the record red.
|
||||
lit := (it.ask == .SYSTEM_SOUND && record_audio_on(*screen_record.recorder, .SYSTEM)) || (it.ask == .MICROPHONE && record_audio_on(*screen_record.recorder, .MICROPHONE));
|
||||
fill(r, ifx lit then cast(u32) RECORD_UI_RED else ifx it_index == record_ui.hovered then cast(u32) RECORD_UI_HOVER else cast(u32) RECORD_UI_BUTTON);
|
||||
SetTextColor(dc, ifx it.ask == .DISCARD then cast(u32) RECORD_UI_AMBER else cast(u32) RECORD_UI_TEXT);
|
||||
wide := utf8_to_wide(it.label,, temp);
|
||||
count: s32;
|
||||
|
||||
@ -732,7 +732,10 @@ screen_record_tests :: () {
|
||||
m, parsed := mcap_parse(cast([] u8) read_entire_file(path,, temp));
|
||||
expect(parsed, "it reads back");
|
||||
if parsed {
|
||||
expect(m.original.count == mp4_size && ends_with(m.original_name, ".mp4"), "the MP4 is its original: % of % bytes, %", m.original.count, mp4_size, m.original_name);
|
||||
// Recorded fragmented, it goes in a regular MP4: the same frames, its header first.
|
||||
original := cast(string) m.original;
|
||||
expect(ends_with(m.original_name, ".mp4") && slice(original, 4, 4) == "ftyp" && find_index_from_left(original, "moof") < 0 && find_index_from_left(original, "moov") < find_index_from_left(original, "mdat"),
|
||||
"the MP4, made regular, is its original: % (% bytes, % recorded)", m.original_name, m.original.count, mp4_size);
|
||||
expect(m.rendered_kind == .OWN, "a frame of it is its rendered image");
|
||||
poster, decoded := image_decode(m.rendered);
|
||||
expect(decoded && poster.width == WIDTH && poster.height == HEIGHT, "at the recording's size");
|
||||
@ -779,6 +782,67 @@ screen_record_tests :: () {
|
||||
}
|
||||
file_delete(leftover_path);
|
||||
}
|
||||
|
||||
// With sound: an AAC stream beside the video, kept through the remux, and capture.json says so.
|
||||
encoder = .{};
|
||||
opened, open_error = encoder_open(*encoder, mp4, WIDTH, HEIGHT, 30, "", audio = true);
|
||||
expect(opened && encoder.audio != null, "an encoder with sound opened: %", open_error);
|
||||
if opened {
|
||||
tone := NewArray(encoder.audio_frame_size * 2, float,, temp);
|
||||
for frame: 0..29 encoder_write(*encoder, pixels.data, frame * 3000);
|
||||
for block: 0..46 { // about a second of 440 Hz
|
||||
for i: 0..encoder.audio_frame_size - 1 {
|
||||
v := sin(cast(float) (block * encoder.audio_frame_size + i) * 2 * PI * 440 / 48000) * 0.5;
|
||||
tone[i * 2], tone[i * 2 + 1] = v, v;
|
||||
}
|
||||
encoder_write_audio(*encoder, tone.data, block * encoder.audio_frame_size);
|
||||
}
|
||||
encoder_close(*encoder);
|
||||
sounding: Screen_Capture_Job;
|
||||
sounding.video = true;
|
||||
sounding.recording = mp4;
|
||||
sounding.date = "2026-10-10 08:30:00";
|
||||
sounding.audio = "Speakers (test)";
|
||||
sounding.shot.width, sounding.shot.height = WIDTH, HEIGHT;
|
||||
sounding_path, sounding_error := screen_record_write(*sounding);
|
||||
expect(!sounding_error, "a recording with sound is written: %", sounding_error);
|
||||
if !sounding_error {
|
||||
m4, parsed4 := mcap_parse(cast([] u8) read_entire_file(sounding_path,, temp));
|
||||
expect(parsed4 && find_index_from_left(cast(string) m4.original, "mp4a") >= 0, "its MP4 has the AAC stream (mp4a)");
|
||||
expect(parsed4 && contains(m4.text, "Speakers (test)"), "capture.json names the sound's source");
|
||||
file_delete(sounding_path);
|
||||
}
|
||||
}
|
||||
|
||||
// One a crash cut short: a frame every 100 ms for 5 s makes fragments at 0, 2 and 4 s (a
|
||||
// keyframe each 2 s, however few the frames); cut at the last, as a crash would, it goes into the
|
||||
// library as a regular MP4 (header first, no fragments) of the 4 s before the cut.
|
||||
encoder = .{};
|
||||
encoder_open(*encoder, mp4, WIDTH, HEIGHT, 30, "");
|
||||
for frame: 0..49 encoder_write(*encoder, pixels.data, frame * 9000);
|
||||
encoder_close(*encoder);
|
||||
whole := read_entire_file(mp4,, temp);
|
||||
last_fragment := -1;
|
||||
for 4..whole.count - 4 if whole[it] == #char "m" && slice(whole, it, 4) == "moof" last_fragment = it - 4;
|
||||
expect(last_fragment > 0, "the recording is fragmented");
|
||||
write_entire_file(mp4, slice(whole, 0, last_fragment));
|
||||
crashed: Screen_Capture_Job;
|
||||
crashed.video = true;
|
||||
crashed.recording = mp4;
|
||||
crashed.date = "2026-10-10 07:48:52";
|
||||
crashed_path, crashed_error := screen_record_write(*crashed);
|
||||
expect(!crashed_error, "a cut recording is written: %", crashed_error);
|
||||
if !crashed_error {
|
||||
m3, parsed3 := mcap_parse(cast([] u8) read_entire_file(crashed_path,, temp));
|
||||
original := cast(string) m3.original;
|
||||
moov := find_index_from_left(original, "moov");
|
||||
mdat := find_index_from_left(original, "mdat");
|
||||
expect(parsed3 && moov > 0 && moov < mdat && find_index_from_left(original, "moof") < 0, "a regular MP4, its header first");
|
||||
crashed_entry := catalog_load(crashed_path);
|
||||
expect(crashed_entry && crashed_entry.duration > 3.5 && crashed_entry.duration < 4.5, "the 4 s before the cut: %", ifx crashed_entry then crashed_entry.duration else -1);
|
||||
if crashed_entry catalog_entry_free(crashed_entry);
|
||||
file_delete(crashed_path);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -311,6 +311,8 @@ App_User_Settings :: struct {
|
||||
video_fps: s64 = VIDEO_FPS_DEFAULT;
|
||||
video_cursor := true;
|
||||
video_capture: string;
|
||||
video_system_audio := true; // what the computer plays
|
||||
video_microphone: bool; // off unless asked for
|
||||
|
||||
// The [hotkeys] lines. Only meaningful when the section was seen; see hotkeys_install.
|
||||
hotkey_bindings: [..] Hotkey_Binding;
|
||||
@ -380,12 +382,12 @@ app_load_user_settings_file :: () -> write_back: bool {
|
||||
if hotkeys_missing.count || global_missing.count return true;
|
||||
|
||||
#if OS == .LINUX seen.global_hotkeys = true; // not written there: Linux has no global hotkeys yet
|
||||
return !(seen.theme && seen.width && seen.height && seen.font_size && seen.font_scale && seen.event_log && seen.hotkeys && seen.global_hotkeys && seen.ocr_model && seen.video_fps && seen.video_cursor && seen.video_capture);
|
||||
return !(seen.theme && seen.width && seen.height && seen.font_size && seen.font_scale && seen.event_log && seen.hotkeys && seen.global_hotkeys && seen.ocr_model && seen.video_fps && seen.video_cursor && seen.video_capture && seen.video_system_audio && seen.video_microphone);
|
||||
}
|
||||
|
||||
Settings_Seen :: struct {
|
||||
theme, width, height, font_size, font_scale, event_log, hotkeys, global_hotkeys, ocr_model: bool;
|
||||
video_fps, video_cursor, video_capture: bool;
|
||||
video_fps, video_cursor, video_capture, video_system_audio, video_microphone: bool;
|
||||
}
|
||||
|
||||
// Of the file's bytes as last loaded or written, so a watcher event for our own save, or for an
|
||||
@ -520,6 +522,10 @@ settings_parse :: (path: string, file_data: string, out: *App_User_Settings) ->
|
||||
seen.video_fps = settings_read_int(*out.video_fps, path, line_number, key, value, VIDEO_FPS_MIN, VIDEO_FPS_MAX);
|
||||
} else if section == "[video]" && key == "cursor" {
|
||||
seen.video_cursor = settings_read_bool(*out.video_cursor, path, line_number, key, value);
|
||||
} else if section == "[video]" && key == "system_audio" {
|
||||
seen.video_system_audio = settings_read_bool(*out.video_system_audio, path, line_number, key, value);
|
||||
} else if section == "[video]" && key == "microphone" {
|
||||
seen.video_microphone = settings_read_bool(*out.video_microphone, path, line_number, key, value);
|
||||
} else if section == "[video]" && key == "capture" {
|
||||
capture := to_lower_copy(settings_unquote(value),, temp);
|
||||
if capture == "auto" || capture == "dxgi" || capture == "wgc" {
|
||||
@ -664,6 +670,7 @@ app_save_user_settings_file :: () -> bool {
|
||||
if seen.event_log app_user_settings.event_log_capacity = on_disk.event_log_capacity;
|
||||
if seen.video_fps app_user_settings.video_fps = on_disk.video_fps;
|
||||
if seen.video_cursor app_user_settings.video_cursor = on_disk.video_cursor;
|
||||
// video_system_audio and video_microphone are the app's: the recording toolbar's buttons set them.
|
||||
if seen.video_capture {
|
||||
free(app_user_settings.video_capture,, context.default_allocator);
|
||||
app_user_settings.video_capture = copy_string(on_disk.video_capture,, context.default_allocator);
|
||||
@ -691,10 +698,13 @@ app_save_user_settings_file :: () -> bool {
|
||||
append(*builder, "[video]\n");
|
||||
append(*builder, "# Screen recording (Ctrl+PrintScreen; Windows). fps: frames a second at most, written only when the\n");
|
||||
append(*builder, "# screen changes. cursor: true or false. capture: auto (Desktop Duplication, else\n");
|
||||
append(*builder, "# Windows.Graphics.Capture), dxgi or wgc.\n");
|
||||
append(*builder, "# Windows.Graphics.Capture), dxgi or wgc. system_audio: what the computer plays; microphone: the\n");
|
||||
append(*builder, "# default input device. Each true or false.\n");
|
||||
print_to_builder(*builder, "fps: %\n", app_user_settings.video_fps);
|
||||
print_to_builder(*builder, "cursor: %\n", ifx app_user_settings.video_cursor then "true" else "false");
|
||||
print_to_builder(*builder, "capture: %\n\n", ifx app_user_settings.video_capture.count then app_user_settings.video_capture else "auto");
|
||||
print_to_builder(*builder, "capture: %\n", ifx app_user_settings.video_capture.count then app_user_settings.video_capture else "auto");
|
||||
print_to_builder(*builder, "system_audio: %\n", ifx app_user_settings.video_system_audio then "true" else "false");
|
||||
print_to_builder(*builder, "microphone: %\n\n", ifx app_user_settings.video_microphone then "true" else "false");
|
||||
append(*builder, "[logging]\n");
|
||||
append(*builder, "# Entries the Event Log keeps in memory. Past it the oldest are dropped; the log file keeps every line.\n");
|
||||
print_to_builder(*builder, "event_log_capacity: %\n", app_user_settings.event_log_capacity);
|
||||
|
||||
@ -20,6 +20,12 @@
|
||||
// WDA_EXCLUDEFROMCAPTURE: is it out of the recording, and the other in?
|
||||
// -animate moves a square across the region the whole time, so the screen changes every
|
||||
// frame: for measuring throughput.
|
||||
// -remux IN OUT a recording's fragmented MP4 made a regular one, as the app does, and nothing else.
|
||||
// -audio WHICH records sound too: system (what the computer plays), mic, or both.
|
||||
// -mic-toggle the microphone turned on 3 s in, and off at 7 s, as the toolbar does.
|
||||
// -system-toggle likewise for what the computer plays.
|
||||
// -sync-test every 2 s a white square flashes on the region and a 1 kHz beep plays; then the
|
||||
// flashes and beeps are found in the file and their offsets given, and how they drift.
|
||||
// -check FILE reads FILE back with FFmpeg and says what it holds, and nothing else: for a
|
||||
// recording cut short by a crash.
|
||||
// -snapshot DIR the last frame captured (captured.png) and a frame decoded back from the file
|
||||
@ -40,6 +46,9 @@ main :: () {
|
||||
list_only, cuda, exclude_test, dpi_unaware, animate, border_test, cursor_test := false, false, false, false, false, false, false;
|
||||
snapshot_folder: string;
|
||||
pause_seconds: float64;
|
||||
sync_test := false;
|
||||
mic_toggle := false;
|
||||
system_toggle := false;
|
||||
i := 1;
|
||||
number :: (index: s64) -> s32 #expand { return cast(s32) string_to_int(`args[index]); }
|
||||
while i < args.count {
|
||||
@ -68,6 +77,16 @@ main :: () {
|
||||
if a == "-simulate-unsupported" { options.testing |= .DXGI_UNSUPPORTED; i += 1; continue; }
|
||||
if a == "-poll" { options.testing |= .WGC_POLL; i += 1; continue; }
|
||||
if a == "-lose" { options.testing |= .LOSE_DUPLICATION; i += 1; continue; }
|
||||
if a == "-sync-test" { sync_test = true; i += 1; continue; }
|
||||
if a == "-mic-toggle" { mic_toggle = true; options.audio_track = true; i += 1; continue; }
|
||||
if a == "-system-toggle" { system_toggle = true; options.audio_track = true; i += 1; continue; }
|
||||
if a == "-audio" && more >= 1 {
|
||||
which := args[i + 1];
|
||||
options.system_audio = which == "system" || which == "both";
|
||||
options.microphone = which == "mic" || which == "both";
|
||||
if !options.system_audio && !options.microphone { print("-audio takes system, mic or both\n"); exit(2); }
|
||||
i += 2; continue;
|
||||
}
|
||||
if a == "-pause" && more >= 1 { pause_seconds = string_to_float64(args[i + 1]); i += 2; continue; }
|
||||
if a == "-cursor-test" { cursor_test = true; i += 1; continue; }
|
||||
if a == "-capture" && more >= 1 {
|
||||
@ -78,6 +97,12 @@ main :: () {
|
||||
else { print("-capture takes auto, dxgi or wgc\n"); exit(2); }
|
||||
i += 2; continue;
|
||||
}
|
||||
if a == "-remux" && more >= 2 {
|
||||
ok, error, packets := record_remux(args[i + 1], args[i + 2]);
|
||||
if !ok { print("Cannot remux %: %\n", args[i + 1], error); exit(1); }
|
||||
print("% packets into %\n", packets, args[i + 2]);
|
||||
exit(0);
|
||||
}
|
||||
if a == "-check" && more >= 1 {
|
||||
image, info, ok, error := video_poster(args[i + 1]);
|
||||
if !ok { print("%: cannot be read: %\n", args[i + 1], error); exit(1); }
|
||||
@ -142,6 +167,15 @@ main :: () {
|
||||
cursor_at = .{region.left + (region.right - region.left) / 2, region.top + (region.bottom - region.top) / 2};
|
||||
SetCursorPos(cursor_at.x, cursor_at.y);
|
||||
}
|
||||
flash: HWND;
|
||||
beep: [] u8;
|
||||
next_flash: float64 = 1;
|
||||
if sync_test {
|
||||
beep = beep_wav();
|
||||
side := min(cast(s32) 200, (region.right - region.left) / 3, (region.bottom - region.top) / 3);
|
||||
flash = square_window("record-probe flash", 0x00FFFFFF, region.left + (region.right - region.left - side) / 2, region.top + (region.bottom - region.top - side) / 2, side);
|
||||
ShowWindow(flash, SW_HIDE);
|
||||
}
|
||||
mover: HWND;
|
||||
if animate mover = square_window("record-probe animate", 0x0000A5FF, region.left, region.top, 120);
|
||||
|
||||
@ -166,6 +200,27 @@ main :: () {
|
||||
print(" % s: % captured, % sent (% repeats), % written, % dropped, % lost, % KB\n",
|
||||
FF(to_float64_seconds(last_print - start), 0), s.captured, s.emitted, s.repeats, s.written, s.dropped, s.lost, s.bytes / 1024);
|
||||
}
|
||||
for kind: Audio_Source_Kind.[.SYSTEM, .MICROPHONE] {
|
||||
if !(ifx kind == .SYSTEM then system_toggle else mic_toggle) continue;
|
||||
elapsed := to_float64_seconds(current_time_monotonic() - start);
|
||||
should := elapsed >= 3 && elapsed < 7;
|
||||
if should != record_audio_on(*recorder, kind) && (elapsed < 3.5 || (elapsed >= 7 && elapsed < 7.5)) {
|
||||
record_set_audio(*recorder, kind, should);
|
||||
print(" % s: % %\n", FF(elapsed, 1), kind, ifx should then "on" else "off");
|
||||
}
|
||||
failed, why := record_audio_failed(*recorder, kind);
|
||||
if failed print(" % failed: %\n", kind, why);
|
||||
}
|
||||
if flash { // a flash and a beep together every 2 s, the flash for 200 ms
|
||||
elapsed := to_float64_seconds(current_time_monotonic() - start);
|
||||
if elapsed >= next_flash {
|
||||
ShowWindow(flash, SW_SHOWNOACTIVATE);
|
||||
PlaySoundW(beep.data, null, SND_MEMORY | SND_ASYNC);
|
||||
next_flash += 2;
|
||||
} else if elapsed >= next_flash - 2 + 0.2 {
|
||||
ShowWindow(flash, SW_HIDE);
|
||||
}
|
||||
}
|
||||
if pause_seconds > 0 {
|
||||
elapsed := to_float64_seconds(current_time_monotonic() - start);
|
||||
should := elapsed >= 2 && elapsed < 2 + pause_seconds;
|
||||
@ -182,6 +237,7 @@ main :: () {
|
||||
|
||||
last, width, height := record_last_frame(*recorder);
|
||||
if mover DestroyWindow(mover);
|
||||
if flash DestroyWindow(flash);
|
||||
if cursor_test SetCursorPos(saved_cursor.x, saved_cursor.y);
|
||||
stopped, stop_error := record_stop(*recorder);
|
||||
s := recorder.stats;
|
||||
@ -201,6 +257,29 @@ main :: () {
|
||||
print(" each: copy % (per image), read back % , encode % (per frame)\n",
|
||||
ms_each(s.copy_ticks, s.captured), ms_each(s.readback_ticks, s.emitted), ms_each(s.encode_ticks, s.written));
|
||||
print(" file: % KB, % MB a minute\n", file_size / 1024, FF(cast(float64) file_size / (1024 * 1024) / s.seconds * 60, 1));
|
||||
if options.system_audio || options.microphone || options.audio_track {
|
||||
print(" sound: %; % AAC frames%\n", ifx s.audio then s.audio else "none opened", s.audio_frames, ifx s.audio_errors then tprint(" (%)", s.audio_errors) else "");
|
||||
sound := sound_report(options.path);
|
||||
if sound.found print(" in file: % s of sound, % dB RMS, % onsets\n", FF(sound.seconds, 2), FF(sound.rms_db, 1), sound.onsets.count);
|
||||
else print(" in file: no sound stream %\n", sound.error);
|
||||
if sync_test && sound.found {
|
||||
flashes := flash_times(options.path);
|
||||
print(" sync: % flashes, % beeps\n", flashes.count, sound.onsets.count);
|
||||
offsets: [..] float64;
|
||||
offsets.allocator = temp;
|
||||
for flash_time: flashes {
|
||||
best: float64 = 1000000000.0;
|
||||
for sound.onsets if abs(it - flash_time) < abs(best) best = it - flash_time;
|
||||
if abs(best) < 0.5 array_add(*offsets, best);
|
||||
}
|
||||
if offsets.count {
|
||||
text: [..] string; text.allocator = temp;
|
||||
for offsets array_add(*text, tprint("%", FF(it * 1000, 0)));
|
||||
print(" offsets: beep after flash, ms: %\n", join(..text, " ",, temp));
|
||||
print(" drift: % ms from the first to the last\n", FF((offsets[offsets.count - 1] - offsets[0]) * 1000, 0));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if exclude_test {
|
||||
excluded_rect := window_rect_in(excluded, region);
|
||||
@ -398,6 +477,134 @@ FF :: (value: float64, decimals: s64) -> FormatFloat #expand {
|
||||
SetProcessDpiAwarenessContext :: (value: HANDLE) -> BOOL #foreign user32_dll;
|
||||
SetWindowDisplayAffinity :: (window: HWND, affinity: u32) -> BOOL #foreign user32_dll;
|
||||
SetCursorPos :: (x: s32, y: s32) -> BOOL #foreign user32_dll;
|
||||
|
||||
// ---------------------------------------------------------------- sound and sync
|
||||
|
||||
// A beep of 1 kHz for 150 ms, as a WAV in memory, for PlaySound.
|
||||
beep_wav :: () -> [] u8 {
|
||||
RATE :: 48000;
|
||||
COUNT :: RATE * 150 / 1000;
|
||||
data := NewArray(44 + COUNT * 2, u8,, context.default_allocator);
|
||||
put32 :: (at: s64, v: u32) #expand { for 0..3 `data[at + it] = cast,trunc(u8) (v >> (8 * it)); }
|
||||
put16 :: (at: s64, v: u16) #expand { `data[at] = cast,trunc(u8) v; `data[at + 1] = cast,trunc(u8) (v >> 8); }
|
||||
memcpy(data.data, "RIFF".data, 4); put32(4, cast(u32) (36 + COUNT * 2)); memcpy(data.data + 8, "WAVEfmt ".data, 8);
|
||||
put32(16, 16); put16(20, 1); put16(22, 1); put32(24, RATE); put32(28, RATE * 2); put16(32, 2); put16(34, 16);
|
||||
memcpy(data.data + 36, "data".data, 4); put32(40, COUNT * 2);
|
||||
for i: 0..COUNT - 1 {
|
||||
v := cast(s16) (sin(cast(float) i * 2 * PI * 1000 / RATE) * 20000);
|
||||
put16(44 + i * 2, cast,no_check(u16) v);
|
||||
}
|
||||
return data;
|
||||
}
|
||||
|
||||
// What the file's sound holds: its length, its level, and when it goes from quiet to loud (onsets).
|
||||
Sound_Report :: struct { found: bool; seconds: float64; rms_db: float64; onsets: [..] float64; error: string; }
|
||||
|
||||
sound_report :: (path: string) -> Sound_Report {
|
||||
report: Sound_Report;
|
||||
report.onsets.allocator = temp;
|
||||
format: *AVFormatContext;
|
||||
if avformat_open_input(*format, temp_c_string(path), null, null) < 0 { report.error = "cannot open"; return report; }
|
||||
defer avformat_close_input(*format);
|
||||
avformat_find_stream_info(format, null);
|
||||
index := -1;
|
||||
for 0..format.nb_streams - 1 if format.streams[it].codecpar.codec_type == .AUDIO { index = it; break; }
|
||||
if index < 0 return report;
|
||||
report.found = true;
|
||||
stream := format.streams[index];
|
||||
codec := avcodec_find_decoder(stream.codecpar.codec_id);
|
||||
c := avcodec_alloc_context3(codec);
|
||||
defer avcodec_free_context(*c);
|
||||
avcodec_parameters_to_context(c, stream.codecpar);
|
||||
if avcodec_open2(c, codec, null) < 0 { report.error = "no decoder"; return report; }
|
||||
packet := av_packet_alloc(); defer av_packet_free(*packet);
|
||||
frame := av_frame_alloc(); defer av_frame_free(*frame);
|
||||
|
||||
WINDOW :: 480; // 10 ms at 48 kHz
|
||||
sum_all: float64; count_all: s64;
|
||||
window_sum: float64; window_count: s64; window_start: float64 = -1;
|
||||
quiet_for: s64 = 100; // windows in a row below the threshold
|
||||
first_time: float64 = -1; last_time: float64;
|
||||
handle :: () #expand {
|
||||
while avcodec_receive_frame(`c, `frame) >= 0 {
|
||||
t := cast(float64) `frame.pts * `stream.time_base.num / `stream.time_base.den;
|
||||
if `first_time < 0 `first_time = t;
|
||||
left := cast(*float) `frame.data[0];
|
||||
for i: 0..`frame.nb_samples - 1 {
|
||||
v := cast(float64) left[i];
|
||||
`sum_all += v * v; `count_all += 1;
|
||||
if `window_start < 0 `window_start = t + cast(float64) i / 48000;
|
||||
`window_sum += v * v; `window_count += 1;
|
||||
if `window_count == WINDOW {
|
||||
rms := sqrt(`window_sum / WINDOW);
|
||||
if rms > 0.05 {
|
||||
if `quiet_for >= 20 array_add(*`report.onsets, `window_start); // after 200 ms of quiet
|
||||
`quiet_for = 0;
|
||||
} else {
|
||||
`quiet_for += 1;
|
||||
}
|
||||
`window_sum, `window_count, `window_start = 0, 0, -1;
|
||||
}
|
||||
}
|
||||
`last_time = t + cast(float64) `frame.nb_samples / 48000;
|
||||
}
|
||||
}
|
||||
while av_read_frame(format, packet) >= 0 {
|
||||
if packet.stream_index == index { avcodec_send_packet(c, packet); handle(); }
|
||||
av_packet_unref(packet);
|
||||
}
|
||||
avcodec_send_packet(c, null);
|
||||
handle();
|
||||
report.seconds = last_time - max(first_time, 0);
|
||||
report.rms_db = ifx count_all && sum_all > 0 then 10 * log(sum_all / count_all) / log(10.0) else -200;
|
||||
return report;
|
||||
}
|
||||
|
||||
// When the video's middle turns white: the sync test's flashes, as times in seconds.
|
||||
flash_times :: (path: string) -> [..] float64 {
|
||||
times: [..] float64;
|
||||
times.allocator = temp;
|
||||
format: *AVFormatContext;
|
||||
if avformat_open_input(*format, temp_c_string(path), null, null) < 0 return times;
|
||||
defer avformat_close_input(*format);
|
||||
avformat_find_stream_info(format, null);
|
||||
index := -1;
|
||||
for 0..format.nb_streams - 1 if format.streams[it].codecpar.codec_type == .VIDEO { index = it; break; }
|
||||
if index < 0 return times;
|
||||
stream := format.streams[index];
|
||||
codec := avcodec_find_decoder(stream.codecpar.codec_id);
|
||||
c := avcodec_alloc_context3(codec);
|
||||
defer avcodec_free_context(*c);
|
||||
avcodec_parameters_to_context(c, stream.codecpar);
|
||||
if avcodec_open2(c, codec, null) < 0 return times;
|
||||
packet := av_packet_alloc(); defer av_packet_free(*packet);
|
||||
frame := av_frame_alloc(); defer av_frame_free(*frame);
|
||||
was_white := false;
|
||||
handle :: () #expand {
|
||||
while avcodec_receive_frame(`c, `frame) >= 0 {
|
||||
// The luma of the middle 32 x 32 (the first plane in every YUV format the encoders give).
|
||||
w, h := `frame.width, `frame.height;
|
||||
sum: s64;
|
||||
for y: h / 2 - 16..h / 2 + 15 for x: w / 2 - 16..w / 2 + 15 sum += `frame.data[0][y * `frame.linesize[0] + x];
|
||||
white := sum / 1024 > 200;
|
||||
if white && !`was_white array_add(*`times, cast(float64) `frame.best_effort_timestamp * `stream.time_base.num / `stream.time_base.den);
|
||||
`was_white = white;
|
||||
}
|
||||
}
|
||||
while av_read_frame(format, packet) >= 0 {
|
||||
if packet.stream_index == index { avcodec_send_packet(c, packet); handle(); }
|
||||
av_packet_unref(packet);
|
||||
}
|
||||
avcodec_send_packet(c, null);
|
||||
handle();
|
||||
return times;
|
||||
}
|
||||
|
||||
PlaySoundW :: (sound: *void, module: *void, flags: u32) -> s32 #foreign winmm_dll;
|
||||
winmm_dll :: #library,system "winmm";
|
||||
SND_ASYNC :: 0x1;
|
||||
SND_MEMORY :: 0x4;
|
||||
|
||||
user32_dll :: #library,system "user32";
|
||||
|
||||
#import "Basic";
|
||||
@ -410,3 +617,4 @@ user32_dll :: #library,system "user32";
|
||||
#import "Screen_Record";
|
||||
#import "Image_Decode";
|
||||
#import "stb_image_write";
|
||||
#import "ffmpeg";
|
||||
|
||||
@ -68,7 +68,7 @@
|
||||
|
||||
<header>
|
||||
<h1>Screen video capture in mcap</h1>
|
||||
<p>Planning document · 2026-10-10 · for mcap after 0.1.1 (still captures with PrintScreen) · <strong>phases 1 to 4 built, mcap 0.1.6</strong></p>
|
||||
<p>Planning document · 2026-10-10 · for mcap after 0.1.1 (still captures with PrintScreen) · <strong>all five phases built, mcap 0.1.8</strong></p>
|
||||
</header>
|
||||
|
||||
<div class="summary">
|
||||
@ -83,6 +83,7 @@
|
||||
<li><span class="tag decided">Done</span> <strong>Phase 1</strong> (2026-10-10, mcap 0.1.3): <code>modules/Screen_Record</code> and <code>record-probe</code>. 60 s of 4K at 30 fps with NVENC and no dropped frames. The two-GPU failure couldn't be reproduced, because this laptop's screens are currently driven by the NVIDIA GPU (see <a href="#phases">phase 1's result</a>).</li>
|
||||
<li><span class="tag decided">Done</span> <strong>Phase 2</strong> (2026-10-10, mcap 0.1.4): the Windows.Graphics.Capture fallback. Automatic mode falls back to it on a simulated <code>DXGI_ERROR_UNSUPPORTED</code>. It records 4K at 60 fps with nothing dropped, draws the cursor exactly as Desktop Duplication's code does, and the yellow border can be turned off on Windows 11.</li>
|
||||
<li><span class="tag decided">Done</span> <strong>Phase 3</strong> (2026-10-10, mcap 0.1.5): recording in the app. Ctrl+PrintScreen opens the overlay's video mode; Ctrl+PrintScreen again stops; the recording becomes a video <code>.mcap</code> in Screen Captures. In the app, Desktop Duplication first failed with <code>DXGI_ERROR_UNSUPPORTED</code>: only the main thread was DPI aware. The capture thread now makes itself aware.</li>
|
||||
<li><span class="tag decided">Done</span> <strong>Phase 5</strong> (2026-10-10, mcap 0.1.8): sound. What the computer plays, and the microphone if asked for, are mixed in sync with the video into an AAC stream. A beep matched to a flash every 2 s stays within about 20 ms, without drifting.</li>
|
||||
<li><span class="tag decided">Done</span> <strong>Phase 4</strong> (2026-10-10, mcap 0.1.6): the border and toolbar, both kept out of the recording; pause, with no gap in the video; discard, confirmed in the toolbar; recovery when the screen is lost; leftover recordings offered at startup; the <code>[video]</code> settings.</li>
|
||||
<li><span class="tag later">Later</span> Audio is phase 5. Recording one window, and regions that span two screens, come after.</li>
|
||||
</ul>
|
||||
@ -434,6 +435,12 @@ Captured_Frame :: struct {
|
||||
<li>Leftover recordings offered at startup, in a popup in the main window.</li>
|
||||
<li>The Win32 GDI bindings moved to <code>src/base/platform/win32_gdi_bindings.jai</code>, shared by the overlay and the toolbar.</li>
|
||||
</ul></li>
|
||||
<li><strong>Found by hand: a killed recording's timestamps were wrong</strong> (mcap 0.1.7). There were two causes:
|
||||
<ul>
|
||||
<li><strong>No length or index.</strong> A fragmented MP4's header gives its length as 0, and one cut short never gets the index a clean stop writes at its end, so players showed its time and seeked badly. Every recording, cut short or not, is now rewritten as a regular MP4 on its way into the library (<code>record_remux</code>): its packets copied as they are, with one header first holding its length and an index of every frame. Nothing is re-encoded.</li>
|
||||
<li><strong>Keyframes counted in frames.</strong> They came every 60 frames, and with frames written only on change, 60 frames can span minutes. So a fragment, and what a crash loses, could span minutes too, not the 2 s section 5 promised. A keyframe is now forced every 2 s of time.</li>
|
||||
</ul>
|
||||
Checked: your recording's MP4, cut mid-fragment, rewrites to a regular MP4 of the 76 whole frames (5.2 s) with positive frame durations. A 12 s mostly still recording has fragments at 0, 2.0, 4.1, 6.1, 8.1, 10.2 and 12.1 s. The app killed about 10 s into a recording leaves 8.1 s. A test cuts a recording at its last fragment, as a crash would, and checks it goes in as a regular MP4 of the time before the cut.</li>
|
||||
<li><strong>Kept out of the recording:</strong> a whole-screen recording with the app (<code>-record 6</code>), where the toolbar sits inside the recorded screen. The frame stored in its <code>.mcap</code> has no toolbar. With <code>-record-ui-visible</code> and a region, a screenshot shows both drawn as designed: the frame just outside the region, and the toolbar below it at its right.</li>
|
||||
<li><strong>Pause:</strong> 16 s with a 10 s pause from 2 s in gives a 6.2 s video at a steady 30 fps, with Desktop Duplication and with Windows.Graphics.Capture alike. The pause is cut out, with no gap.</li>
|
||||
<li><strong>Losing the screen</strong> (<code>-lose</code>, which loses Desktop Duplication every 3 s through the real recovery path): 3 losses in 10 s, 304 frames, 30 fps throughout. A real UAC prompt or resolution change is still to try by hand.</li>
|
||||
@ -460,9 +467,48 @@ Captured_Frame :: struct {
|
||||
</div>
|
||||
|
||||
<div class="phase">
|
||||
<h3>Phase 5: Audio</h3>
|
||||
<h3>Phase 5: Audio <span class="tag decided">Done</span></h3>
|
||||
<p>Section 8: <code>swresample</code> bindings, loopback and microphone, mixing, AAC.</p>
|
||||
<p><strong>Done when:</strong> a 10-minute recording with a video playing and someone talking stays in sync to within 50 ms at the end, and silence doesn't shift it.</p>
|
||||
<div class="note"><strong>Result (2026-10-10).</strong>
|
||||
<ul>
|
||||
<li><strong>Built:</strong>
|
||||
<ul>
|
||||
<li><code>modules/Screen_Record/audio_source.jai</code>: WASAPI for both sources. What the computer plays comes in loopback from the default output device; the microphone is the default input device.</li>
|
||||
<li>An audio thread in <code>module.jai</code> that mixes them.</li>
|
||||
<li>An AAC stream in <code>encoder.jai</code>, with the two threads' packets meeting in the file under a lock.</li>
|
||||
<li><code>system_audio</code> and <code>microphone</code> in <code>[video]</code>, defaulting to true and false.</li>
|
||||
</ul></li>
|
||||
<li><strong>Sync, by measurement:</strong> <code>record-probe -sync-test</code> flashes a white square and plays a 1 kHz beep together every 2 s, then finds both in the file.
|
||||
<ul>
|
||||
<li>20 s with system sound: 10 flashes, 10 beeps, each beep within 0 to 21 ms of its flash, with no drift (−10 ms first to last, within the spread).</li>
|
||||
<li>Both sources mixed: within 5 to 18 ms.</li>
|
||||
<li>With a 6 s pause: sound and video lose the same 6 s (14.1 s of sound, 14.2 s of video), and the beeps after the pause line up as before.</li>
|
||||
<li>3 minutes with system sound: 92 flashes, 90 beeps found. 85 beeps were within 3 to 24 ms of their flash, and from the first to the last the offset moved −14 ms: no drift. Five were 53 to 147 ms off, in two clusters about 40 s and 60 s in, near where the two beeps went missing. That points to Windows playing the test beep late or not at all then, but it isn't proven; a 10-minute run with someone talking, as the done-when asks, is still to do by hand.</li>
|
||||
</ul>
|
||||
The offsets include Windows' own delay in playing the beep, so they are an upper bound on the true offset.</li>
|
||||
<li><strong>In the app:</strong> <code>-record 6</code> recorded the monitor's speakers. The <code>.mcap</code>'s MP4 has its AAC stream, and its <code>capture.json</code> has <code>Audio: true</code> and the device's name. Your settings file gained <code>system_audio</code> and <code>microphone</code>.</li>
|
||||
<li><strong>Tests:</strong> 4 more checks, 30 in the recording group. An MP4 made with sound by the module's encoder keeps its AAC stream through the remux into the library, and <code>capture.json</code> names its source. Every suite passes.</li>
|
||||
</ul>
|
||||
<p><strong>Changes from the plan:</strong></p>
|
||||
<ul>
|
||||
<li><strong>No <code>swresample</code>.</strong> WASAPI converts any device to 48 kHz stereo float itself (<code>AUTOCONVERTPCM</code>), so no bindings were generated. A device that won't convert is taken as it is if it gives 48 kHz float, its channels made stereo here.</li>
|
||||
<li><strong>How the timeline holds together:</strong>
|
||||
<ul>
|
||||
<li>Each packet is placed by the time Windows says it was heard, on the video's own timeline: the same time 0, less the same pauses. <code>record_pause</code> now keeps one account of time paused for both threads.</li>
|
||||
<li>A packet within 30 ms of where its source left off follows on exactly, so timing jitter doesn't leave clicks.</li>
|
||||
<li>A source that sends nothing, as loopback does while nothing plays, is silence, and the mix moves on without it 200 ms late.</li>
|
||||
</ul></li>
|
||||
<li><strong>The microphone in the toolbar</strong> (added in mcap 0.1.9, on request).
|
||||
<ul>
|
||||
<li><strong>Mic: on / off</strong> turns the microphone on and off mid-recording (<code>record_set_microphone</code>), and its state is kept as <code>microphone</code> in <code>[video]</code> for the next recording.</li>
|
||||
<li>The microphone is opened only while on, so Windows' microphone-in-use indicator shows only then.</li>
|
||||
<li>An MP4 can't gain a sound stream once its header is written, so the app's recordings always have one (<code>audio_track</code>), silent while nothing is on.</li>
|
||||
<li>Checked with <code>record-probe -mic-toggle</code>, which turns the microphone on at 3 s and off at 7 s. With system sound, and with nothing else on, the sound track runs the whole recording without a break.</li>
|
||||
</ul>
|
||||
In mcap 0.1.10, <strong>System sound: on / off</strong> joined it and works the same way, saved as <code>system_audio</code>. With <code>record-probe -system-toggle -sync-test</code> (system sound on at 3 s, off at 7 s), only the beeps at 3 and 5 s are in the file, 8 to 12 ms from their flashes, and the track runs the whole recording.</li>
|
||||
<li><strong>A lost device</strong> (a headset unplugged, the default changed) is opened again from the new default once a second.</li>
|
||||
</ul></div>
|
||||
</div>
|
||||
|
||||
<h2 id="risks">10. Risks</h2>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user