MusaMahmood/mcap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

[0.1.34] macOS phase 7: screen recording with Opt+Cmd+Shift+2, through ScreenCaptureKit and VideoToolbox

Browse Source
This commit is contained in:
Musa Mahmood 2026-10-17 05:57:10 -04:00
parent 63a0cdb64e
commit 74b273f0e0
17 changed files with 2142 additions and 820 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@ -220,7 +220,7 @@ On macOS (Cmd+Shift+3, 4 and 5 are the system's own screenshot keys):
### Recording the screen ### Recording the screen
**Ctrl+PrintScreen** records a region of one screen as a video: **Ctrl+PrintScreen** (**Opt+Cmd+Shift+2** on macOS) records a region of one screen as a video:
1. **The screens freeze**, as for a still, with the outline and magnifier in red. 1. **The screens freeze**, as for a still, with the outline and magnifier in red.
2. **Drag a region**, or click (or press Enter) for the whole screen under the cursor: 2. **Drag a region**, or click (or press Enter) for the whole screen under the cursor:
@ -238,14 +238,15 @@ On macOS (Cmd+Shift+3, 4 and 5 are the system's own screenshot keys):
and **Mic** (each on or off), **Pause** (the time paused is left out of the video), **Stop**, and **Discard**, which asks and **Mic** (each on or off), **Pause** (the time paused is left out of the video), **Stop**, and **Discard**, which asks
first. first.
- **Neither is in the recording.** Neither takes the focus from the window being recorded. - **Neither is in the recording.** Neither takes the focus from the window being recorded.
4. **Ctrl+PrintScreen again**, or Stop, stops it. The video goes into **Screen Captures** beside the stills, 4. **The same key again**, or Stop, stops it. The video goes into **Screen Captures** beside the stills,
as an H.264 MP4 in an `.mcap` with the window it was recorded from, and opens in the Capture window. as an H.264 MP4 in an `.mcap` with the window it was recorded from, and opens in the Capture window.
OCR and labels read a frame from it. OCR and labels read a frame from it.
- **The screen** is taken with DXGI Desktop Duplication, or with Windows.Graphics.Capture where that - **The screen** is taken with DXGI Desktop Duplication, or with Windows.Graphics.Capture where that
cannot run (a laptop with two GPUs, a rotated screen). cannot run (a laptop with two GPUs, a rotated screen). On macOS, with a ScreenCaptureKit stream
(`modules/Screen_Record/macos/capture.m`), which also brings the sound.
- **Encoding** uses NVENC where there is an NVIDIA GPU, else AMD's or Intel's encoder, Media - **Encoding** uses NVENC where there is an NVIDIA GPU, else AMD's or Intel's encoder, Media
Foundation, or libx264. Foundation, or libx264. On macOS, VideoToolbox.
- **Losing nothing:** - **Losing nothing:**
- A recording that stops by itself keeps what it has, and so does quitting while recording. - A recording that stops by itself keeps what it has, and so does quitting while recording.
- When the screen is lost for a moment (a UAC prompt, the lock screen, a resolution change), the - When the screen is lost for a moment (a UAC prompt, the lock screen, a resolution change), the
@ -254,11 +255,13 @@ On macOS (Cmd+Shift+3, 4 and 5 are the system's own screenshot keys):
keeps all but its last two seconds or so. keeps all but its last two seconds or so.
- **Settings:** `[video]` in the settings file: - **Settings:** `[video]` in the settings file:
- `fps` (30) and `cursor` (true). - `fps` (30) and `cursor` (true).
- `capture`: `auto` (Desktop Duplication, else Windows.Graphics.Capture), `dxgi` or `wgc`. - `capture` (Windows): `auto` (Desktop Duplication, else Windows.Graphics.Capture), `dxgi` or `wgc`.
- `system_audio` (true): what the computer plays, from the default output device. The toolbar's - `system_audio` (true): what the computer plays, from the default output device. The toolbar's
System sound button sets it. System sound button sets it.
- `microphone` (false): the default input device. The toolbar's Mic button sets it. - `microphone` (false): the default input device. The toolbar's Mic button sets it.
- Sound is AAC at 160 kb/s. A device unplugged mid-recording is picked up again from the new default. - Sound is AAC at 160 kb/s. A device unplugged mid-recording is picked up again from the new default.
- On macOS the microphone needs macOS 15 or later, and its permission (asked for the first time the
Mic button turns it on).
- **PrintScreen while recording** takes the whole screen under the cursor at once, without freezing - **PrintScreen while recording** takes the whole screen under the cursor at once, without freezing
the screens, which would show in the recording. the screens, which would show in the recording.
- **The key** is `record_region` in `[global_hotkeys]`. - **The key** is `record_region` in `[global_hotkeys]`.

2
assets/data/default_global_hotkeys_macos.config
View File

@ -6,4 +6,4 @@
none bring_to_foreground none bring_to_foreground
Cmd-Shift-2 capture_region Cmd-Shift-2 capture_region
none record_region Opt-Cmd-Shift-2 record_region

2
check_targets.jai
View File

@ -52,7 +52,7 @@ APP_NAME :: "mcap";
APP_FOLDER_NAME :: "mcap"; APP_FOLDER_NAME :: "mcap";
GIT_BRANCH :: "check"; GIT_BRANCH :: "check";
GIT_HASH :: "-"; GIT_HASH :: "-";
VERSION :: "0.1.33"; VERSION :: "0.1.34";
BUILD_DATE :: "-"; BUILD_DATE :: "-";
DEBUG :: true; DEBUG :: true;
COMPILE_TESTS :: true; COMPILE_TESTS :: true;

3
first.jai
View File

@ -8,7 +8,7 @@ APP_NAME :: "mcap";
APP_FOLDER_NAME :: "mcap"; APP_FOLDER_NAME :: "mcap";
APP_BUNDLE_IDENTIFIER :: "com.example.mcap"; APP_BUNDLE_IDENTIFIER :: "com.example.mcap";
VERSION :: "0.1.33"; VERSION :: "0.1.34";
// Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated // Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated
// builds never write over the ones you run by hand. // builds never write over the ones you run by hand.
build_directory := "bin"; build_directory := "bin";
@ -373,6 +373,7 @@ finish_main_build :: (build: Pending_Build) {
<key>CFBundleVersion</key> <string>%</string> <key>CFBundleVersion</key> <string>%</string>
<key>LSMinimumSystemVersion</key> <string>%</string> <key>LSMinimumSystemVersion</key> <string>%</string>
<key>NSHighResolutionCapable</key> <true/> <key>NSHighResolutionCapable</key> <true/>
<key>NSMicrophoneUsageDescription</key> <string>mcap records the microphone into a screen recording when its Mic button is on.</string>
</dict> </dict>
</plist> </plist>
PLIST PLIST

77
modules/Screen_Record/audio_mix.jai Normal file
View File

@ -0,0 +1,77 @@
// The sound of a recording, mixed: each source's packets put on the recording's timeline and added
// together, and an AAC frame encoded once every source has covered it. Shared by the Windows and
// macOS recorders (windows.jai, macos.jai), whose Recorder each has the fields used here: moved here
// from module.jai unchanged.
#scope_module
// The sound so far, from its base: one AAC frame's worth is encoded at a time.
Audio_Mix :: struct {
samples: [..] float; // interleaved stereo, from base on
base: s64; // the frame (in samples, on the timeline) samples[0] is
next: [2] s64; // each source's next frame, so packets that follow on are not moved by jitter
started: [2] bool;
frame_size: s32;
}
AUDIO_SLACK :: AUDIO_RATE / 5; // 200 ms: how late a source's packets may come and still be in the mix
AUDIO_FOLLOW_ON :: AUDIO_RATE / 33; // 30 ms: a packet that close to where its source left off follows on
// A packet into the mix at its place on the timeline, added to what the other source put there.
audio_mix_add :: (mix: *Audio_Mix, r: *Recorder, source: s64, packet: Audio_Packet) {
at := record_timeline(r, packet.ticks) * AUDIO_RATE / r.frequency;
if mix.started[source] && abs(at - mix.next[source]) <= AUDIO_FOLLOW_ON at = mix.next[source];
mix.started[source] = true;
mix.next[source] = at + packet.frames;
end := (at + packet.frames - mix.base) * AUDIO_CHANNELS;
if end > mix.samples.count {
old := mix.samples.count;
array_resize(*mix.samples, end, initialize = false);
for old..end - 1 mix.samples[it] = 0;
}
if packet.silent || !packet.samples.count return;
for frame: max(at, mix.base)..at + packet.frames - 1 {
into := (frame - mix.base) * AUDIO_CHANNELS;
from := (frame - at) * AUDIO_CHANNELS;
mix.samples[into] += packet.samples[from];
mix.samples[into + 1] += packet.samples[from + 1];
}
}
// Encodes the frames every open source has covered: up to where each has sent, or, for one that has
// sent nothing lately (silence), to AUDIO_SLACK ago. final: everything sent.
audio_mix_encode :: (mix: *Audio_Mix, r: *Recorder, final: bool) -> ok: bool, error: string {
// Paused, the timeline stands still where the pause began.
now := ifx atomic_read(*r.paused) then atomic_read(*r.pause_began) else ticks_now();
now_frame := record_timeline(r, now) * AUDIO_RATE / r.frequency;
ready := now_frame - AUDIO_SLACK;
for source, index: r.audio_sources {
if !source.open continue;
covered := max(mix.next[index], now_frame - AUDIO_SLACK);
ready = min(ready, covered);
}
if final { ready = mix.base; for mix.next ready = max(ready, it); }
size := mix.frame_size;
frame := NewArray(size * AUDIO_CHANNELS, float, initialized = false,, temp);
done: s64;
while mix.base + size <= ready {
for i: 0..frame.count - 1 {
at := done * AUDIO_CHANNELS + i;
value := ifx at < mix.samples.count then mix.samples[at] else 0.0;
frame[i] = clamp(value, -1.0, 1.0);
}
ok, error := encoder_write_audio(*r.encoder, frame.data, mix.base);
if !ok return false, error;
mix.base += size;
done += size;
lock(*r.mutex); r.stats.audio_frames += 1; unlock(*r.mutex);
}
// What was encoded leaves the front.
gone := min(done * AUDIO_CHANNELS, mix.samples.count);
if gone {
memcpy(mix.samples.data, mix.samples.data + gone, (mix.samples.count - gone) * size_of(float));
mix.samples.count -= gone;
}
return true, "";
}

17
modules/Screen_Record/encoder.jai
View File

@ -1,6 +1,7 @@
// The encoder thread's half: frames of BGRA pixels to H.264 in a fragmented MP4, with FFmpeg // The encoder thread's half: frames of BGRA pixels to H.264 in a fragmented MP4, with FFmpeg
// (modules/ffmpeg). The first encoder in RECORD_ENCODERS that opens at the size wanted is used: // (modules/ffmpeg). The first encoder in RECORD_ENCODERS that opens at the size wanted is used:
// NVENC, AMD's AMF, Intel's QSV, Windows' Media Foundation, then libx264 on the CPU. NVENC takes the // NVENC, AMD's AMF, Intel's QSV, Windows' Media Foundation, then libx264 on the CPU; on macOS,
// VideoToolbox (Apple silicon's media engine), at a constant quality. NVENC takes the
// BGRA as it is and converts it on the GPU; the others are given NV12 or YUV 4:2:0, converted with // BGRA as it is and converts it on the GPU; the others are given NV12 or YUV 4:2:0, converted with
// swscale (BT.709, limited range, as tagged). // swscale (BT.709, limited range, as tagged).
// //
@ -12,7 +13,8 @@
// span any time: a keyframe is made every RECORD_KEYFRAME_SECONDS of time instead, which bounds what a // span any time: a keyframe is made every RECORD_KEYFRAME_SECONDS of time instead, which bounds what a
// crash loses. remux.jai makes the file a regular MP4 once it is done. // crash loses. remux.jai makes the file a regular MP4 once it is done.
RECORD_ENCODERS :: string.["h264_nvenc", "h264_amf", "h264_qsv", "h264_mf", "libx264"]; #if OS == .MACOS RECORD_ENCODERS :: string.["h264_videotoolbox"]; // Apple's, in every Mac; the macOS FFmpeg has no libx264
else RECORD_ENCODERS :: string.["h264_nvenc", "h264_amf", "h264_qsv", "h264_mf", "libx264"];
RECORD_KEYFRAME_SECONDS :: 2; RECORD_KEYFRAME_SECONDS :: 2;
RECORD_AUDIO_BIT_RATE :: 160_000; RECORD_AUDIO_BIT_RATE :: 160_000;
@ -170,6 +172,7 @@ encoder_close :: (e: *Encoder) -> ok: bool, error: string {
#scope_file #scope_file
RECORD_TIME_BASE :: 90000; // pts ticks a second: MPEG's RECORD_TIME_BASE :: 90000; // pts ticks a second: MPEG's
VIDEOTOOLBOX_QUALITY :: 65; // 0-100: sharp text at a few MB a minute for a screen
encoder_try :: (e: *Encoder, name: string, width: s32, height: s32, fps: s32) -> ok: bool, error: string { encoder_try :: (e: *Encoder, name: string, width: s32, height: s32, fps: s32) -> ok: bool, error: string {
codec := avcodec_find_encoder_by_name(temp_c_string(name)); codec := avcodec_find_encoder_by_name(temp_c_string(name));
@ -203,6 +206,13 @@ encoder_try :: (e: *Encoder, name: string, width: s32, height: s32, fps: s32) ->
case "h264_qsv"; set("global_quality", "23"); set("preset", "medium"); case "h264_qsv"; set("global_quality", "23"); set("preset", "medium");
case "h264_mf"; set("rate_control", "quality"); set("quality", "70"); set("scenario", "display_remoting"); case "h264_mf"; set("rate_control", "quality"); set("quality", "70"); set("scenario", "display_remoting");
case "libx264"; set("preset", "veryfast"); set("crf", "23"); set("forced-idr", "1"); case "libx264"; set("preset", "veryfast"); set("crf", "23"); set("forced-idr", "1");
case "h264_videotoolbox";
// A constant quality (VideoToolbox's 0-1, as 0-100), which Apple silicon takes: FFmpeg's default
// bit rate, 200 kb/s, would blur a screen. Real time, as frames come as they are recorded.
set("realtime", "1");
set("allow_sw", "0");
c.flags |= AV_CODEC_FLAG_QSCALE;
c.global_quality = FF_QP2LAMBDA * VIDEOTOOLBOX_QUALITY;
} }
result := avcodec_open2(c, codec, *options); result := avcodec_open2(c, codec, *options);
av_dict_free(*options); av_dict_free(*options);
@ -231,7 +241,8 @@ encoder_input_format :: (codec: *AVCodec) -> AVPixelFormat {
// The packets an encoder (the video's, unless another is given) has ready, into the file. // The packets an encoder (the video's, unless another is given) has ready, into the file.
encoder_drain :: (e: *Encoder, codec: *AVCodecContext = null, stream: *AVStream = null, packet: *AVPacket = null) -> ok: bool, error: string { encoder_drain :: (e: *Encoder, codec: *AVCodecContext = null, stream: *AVStream = null, packet: *AVPacket = null) -> ok: bool, error: string {
AVERROR_EAGAIN :: -11; #if OS == .MACOS AVERROR_EAGAIN :: -35; // AVERROR(EAGAIN): -EAGAIN, whose errno differs
else AVERROR_EAGAIN :: -11;
if !codec codec, stream, packet = e.codec, e.stream, e.packet; if !codec codec, stream, packet = e.codec, e.stream, e.packet;
while true { while true {
result := avcodec_receive_packet(codec, packet); result := avcodec_receive_packet(codec, packet);

526
modules/Screen_Record/macos.jai Normal file
View File

@ -0,0 +1,526 @@
// The macOS recorder (module.jai has what is shared; windows.jai is the Windows one, which this follows).
// A ScreenCaptureKit stream (modules/Screen_Record/macos/capture.m) of one display's region gives the
// frames, already scaled to the region's pixels and with the cursor drawn in when asked, and the sound:
// what the computer plays, and on macOS 15 and later the microphone, both from the same stream, so
// there is no device to open here. mcap's own sound is left out.
//
// Three threads, as on Windows. The capture thread takes the newest frame, at most fps a second and
// one a second while nothing changes, so players can seek; each carries the host time it was shown,
// less the time paused. The encoder thread hands them to FFmpeg (encoder.jai, with VideoToolbox), the
// oldest dropped when it falls RECORD_QUEUE behind. The audio thread puts each source's packets on the
// same timeline into the mix (audio_mix.jai), and turns the sources on and off in the stream's
// configuration as record_set_audio asks.
//
// Times are the host clock's (mach_absolute_time, CLOCK_UPTIME_RAW) in nanoseconds, which
// ScreenCaptureKit's sample times are on too.
AUDIO_RATE :: 48000;
AUDIO_CHANNELS :: 2;
Audio_Source_Kind :: enum u8 { SYSTEM; MICROPHONE; }
// A source of sound in the stream: open while the stream is set to give it.
Audio_Source :: struct {
kind: Audio_Source_Kind;
name: string; // "System sound", "Microphone"
open: bool;
lost_at: s64; // the mix's; never lost here
}
Audio_Packet :: struct {
samples: [] float; // interleaved stereo, AUDIO_CHANNELS a frame
frames: s64;
ticks: s64; // the host time its first frame was heard, in nanoseconds
silent: bool;
}
Record_Options :: struct {
path: string; // the .mp4 written
display_id: u32; // the display (CGDirectDisplayID)
x, y, w, h: float64; // the region, in the display's points from its top left
width, height: s32; // and in its pixels: rounded down to even here
fps: s32 = 30;
cursor := true;
encoder: string; // "" tries RECORD_ENCODERS
system_audio: bool; // what the computer plays
microphone: bool; // the default input (macOS 15 and later); both at the start: record_set_audio turns them on and off
audio_track: bool; // a sound track whatever is on at the start, so a source can be turned on later
exclude_windows: [] u32; // window numbers kept out of the frames: the border and toolbar
}
Record_Stats :: struct {
source: string; // "ScreenCaptureKit, display 1"
fell_back_from: string; // never, here
encoder: string;
width, height: s32;
captured: s64; // new frames from the stream
pointer_moves: s64;
emitted: s64; // frames sent to be encoded
repeats: s64; // of them, the once-a-second repeat of an unchanged screen
written: s64; // encoded
dropped: s64; // the encoder was too far behind
lost: s64;
paused_seconds: float64; // left out of the video
audio: string; // the sound recorded: "System sound; Microphone", or ""
audio_frames: s64; // AAC frames encoded
audio_errors: string; // sources that could not be turned on
bytes: s64; // encoded
seconds: float64; // from start to stop
}
Recorder :: struct {
options: Record_Options;
stream: *Mcap_Sck_Stream;
encoder: Encoder;
width, height: s32;
frequency: s64 = 1_000_000_000; // ticks a second: nanoseconds
start_ticks: s64;
capture_thread, encoder_thread, audio_thread: Thread;
capture_started, encoder_started, audio_started: bool;
audio_sources: [2] Audio_Source;
audio_open: bool; // the MP4 has sound
audio_on: [2] bool; // each source is wanted (by Audio_Source_Kind): record_set_audio
audio_failed: [2] bool; // it could not be turned on: record_audio_failed says once
zero_ticks: s64; // the recording's time 0
paused_ticks: s64; // time paused so far; set by record_pause
pause_began: s64;
running: bool;
stopping: bool;
paused: bool;
capture_ended: bool;
mutex: Mutex;
ready: Semaphore; // signalled for each frame queued, and once when capture ends
latest: [] u8; // the stream's newest frame, BGRA: the capture thread's
frames: [RECORD_BUFFERS] Record_Frame;
free: [RECORD_BUFFERS] s32;
free_count: s32;
queue: [RECORD_BUFFERS] s32; // oldest first
queue_count: s32;
last_frame: [] u8; // the last frame queued, for record_last_frame
stats: Record_Stats;
error: string; // why it stopped by itself; heap
}
Record_Frame :: struct {
pixels: [] u8; // BGRA, rows packed
ticks: s64;
}
// Starts the stream, the encoder and the file, and starts recording. fall_back is never set: there is
// one way of taking the screen here.
record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: string, fall_back: bool {
r.* = .{};
r.options = options;
r.options.path = copy_string(options.path);
r.options.encoder = copy_string(options.encoder);
r.options.exclude_windows = array_copy(options.exclude_windows);
if r.options.fps <= 0 r.options.fps = 30;
if !record_size_allowed(options.width, options.height)
return false, tprint("% x % is smaller than % x %", options.width, options.height, RECORD_MIN_WIDTH, RECORD_MIN_HEIGHT), false;
// Even both ways: H.264's 4:2:0 colour needs it. The region in points shrinks to match.
r.width = options.width & cast(s32) ~1;
r.height = options.height & cast(s32) ~1;
scale := options.w / options.width;
w := r.width * scale;
h := r.height * scale;
r.stats.width, r.stats.height = r.width, r.height;
init(*r.mutex, "record");
init(*r.ready);
for * r.frames it.pixels = NewArray(cast(s64) r.width * r.height * 4, u8, initialized = false);
for 0..RECORD_BUFFERS - 1 r.free[it] = cast(s32) it;
r.free_count = RECORD_BUFFERS;
r.latest = NewArray(cast(s64) r.width * r.height * 4, u8);
r.last_frame = NewArray(cast(s64) r.width * r.height * 4, u8);
// The microphone only where the stream can give it.
microphone := options.microphone && mcap_sck_microphone_supported() != 0;
if options.microphone && !microphone {
r.stats.audio_errors = copy_string("microphone: it needs macOS 15 or later");
r.audio_failed[Audio_Source_Kind.MICROPHONE] = true;
}
error_text: [512] u8;
excluded := r.options.exclude_windows;
r.stream = mcap_sck_stream_start(options.display_id, options.x, options.y, w, h, r.width, r.height, r.options.fps,
xx ifx options.cursor then 1 else 0, xx ifx options.system_audio then 1 else 0, xx ifx microphone then 1 else 0,
excluded.data, xx excluded.count, error_text.data, error_text.count);
if !r.stream {
error := tprint("ScreenCaptureKit could not record: %", to_string(error_text.data));
record_free(r);
return false, error, false;
}
r.stats.source = sprint("ScreenCaptureKit, display %", options.display_id);
r.start_ticks = ticks_now();
r.zero_ticks = r.start_ticks;
r.audio_sources[0] = .{kind = .SYSTEM, name = "System sound", open = options.system_audio};
r.audio_sources[1] = .{kind = .MICROPHONE, name = "Microphone", open = microphone};
r.audio_on = .[options.system_audio, microphone];
r.audio_open = options.system_audio || microphone || options.audio_track;
record_note_audio(r);
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder, audio = r.audio_open);
if tried log("[record] Encoders that did not open: %", tried);
if !ok {
error = copy_string(error,, temp);
record_free(r);
return false, error, false;
}
r.stats.encoder = copy_string(r.encoder.name);
r.running = true;
thread_init(*r.capture_thread, record_capture_proc, name = "record capture");
r.capture_started = true;
thread_start(*r.capture_thread, r);
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
r.encoder_started = true;
thread_start(*r.encoder_thread, r);
if r.audio_open {
thread_init(*r.audio_thread, record_audio_proc, name = "record audio");
r.audio_started = true;
thread_start(*r.audio_thread, r);
}
return true, "", false;
}
// Stops, finishes the file and frees everything. ok is false when recording stopped by itself on an
// error, or the file could not be finished; what was recorded up to then is in the file all the same.
record_stop :: (r: *Recorder) -> ok: bool, error: string {
if !r.running return false, "not recording";
atomic_swap(*r.stopping, true);
thread_is_done(*r.capture_thread, -1);
thread_is_done(*r.encoder_thread, -1);
if r.audio_started thread_is_done(*r.audio_thread, -1);
r.running = false;
closed, close_error := encoder_close(*r.encoder);
r.stats.seconds = cast(float64) (ticks_now() - r.start_ticks) / r.frequency;
r.stats.bytes = r.encoder.bytes;
error := ifx r.error then copy_string(r.error,, temp) else "";
record_free(r);
if error return false, error;
if !closed return false, close_error;
return true, "";
}
// Pauses or resumes; the time paused is left out of the video, sound and all.
record_pause :: (r: *Recorder, pause: bool) {
if !r.running || atomic_read(*r.paused) == pause return;
now := ticks_now();
if pause {
atomic_write(*r.pause_began, now);
atomic_swap(*r.paused, true);
} else {
atomic_add(*r.paused_ticks, now - atomic_read(*r.pause_began));
atomic_swap(*r.paused, false);
}
}
// Turns a source of sound on or off while recording; the stream is told on the audio thread. False
// when the recording has no sound track to put it in.
record_set_audio :: (r: *Recorder, kind: Audio_Source_Kind, on: bool) -> bool {
if !r.running || !r.audio_open return false;
atomic_swap(*r.audio_on[kind], on);
return true;
}
record_audio_on :: (r: *Recorder, kind: Audio_Source_Kind) -> bool {
return r.running && atomic_read(*r.audio_on[kind]);
}
// Whether a source could not be turned on (it is off again then), and why; once.
record_audio_failed :: (r: *Recorder, kind: Audio_Source_Kind) -> failed: bool, error: string {
if !r.running || !atomic_read(*r.audio_failed[kind]) return false, "";
atomic_swap(*r.audio_failed[kind], false);
lock(*r.mutex);
defer unlock(*r.mutex);
return true, copy_string(r.stats.audio_errors,, temp);
}
// Ticks on the recording's timeline: time 0 at its start, less the time paused.
record_timeline :: (r: *Recorder, ticks: s64) -> s64 {
return ticks - r.zero_ticks - atomic_read(*r.paused_ticks);
}
record_paused :: (r: *Recorder) -> bool {
return r.running && atomic_read(*r.paused);
}
record_stats :: (r: *Recorder) -> Record_Stats {
if !r.running return r.stats;
lock(*r.mutex);
defer unlock(*r.mutex);
return r.stats;
}
// The last frame queued, BGRA (temp), cursor and all.
record_last_frame :: (r: *Recorder) -> pixels: [] u8, width: s32, height: s32 {
if r.running lock(*r.mutex);
pixels := array_copy(r.last_frame,, temp);
if r.running unlock(*r.mutex);
return pixels, r.width, r.height;
}
record_failed :: (r: *Recorder) -> bool, string {
lock(*r.mutex);
defer unlock(*r.mutex);
return r.error.count > 0, r.error;
}
// The host clock, in nanoseconds: ScreenCaptureKit's sample times are on it.
ticks_now :: () -> s64 {
CLOCK_UPTIME_RAW :: 8;
return cast(s64) clock_gettime_nsec_np(CLOCK_UPTIME_RAW);
}
#scope_file
// The sources heard, for the stats: "System sound; Microphone".
record_note_audio :: (r: *Recorder) {
names: [..] string;
names.allocator = temp;
for r.audio_sources if it.open array_add(*names, it.name);
lock(*r.mutex);
free(r.stats.audio);
r.stats.audio = join(..names, "; ");
unlock(*r.mutex);
}
record_free :: (r: *Recorder) {
atomic_swap(*r.stopping, true);
if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; }
if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; }
if r.audio_started { thread_is_done(*r.audio_thread, -1); thread_deinit(*r.audio_thread); r.audio_started = false; }
if r.stream { mcap_sck_stream_stop(r.stream); r.stream = null; }
if r.encoder.format encoder_close(*r.encoder);
for * r.frames array_free(it.pixels);
array_free(r.latest);
array_free(r.last_frame);
r.latest, r.last_frame = .[], .[];
array_free(r.options.exclude_windows);
r.options.exclude_windows = .[];
destroy(*r.ready);
destroy(*r.mutex);
free(r.error);
r.error = "";
}
// Takes the stream's frames as they come, and sends one when its turn is due: at most fps a second
// (a tenth of a frame early counts as on time), and one a second while nothing changes.
record_capture_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
period := r.frequency / r.options.fps;
early := period / 10;
turn := ticks_now();
last_emit: s64 = -r.frequency;
have_image, changed := false, false;
frame_time: s64;
was_paused := false;
error_text: [512] u8;
while !atomic_read(*r.stopping) {
now := ticks_now();
due := ifx changed then turn - early else last_emit + r.frequency;
wait := cast(s32) clamp((due - now) * 1000 / r.frequency, 1, 100);
shown: s64;
got := mcap_sck_next_frame(r.stream, r.latest.data, *shown, wait, error_text.data, error_text.count);
if got < 0 {
record_fail(r, tprint("ScreenCaptureKit stopped: %", to_string(error_text.data)));
break;
}
if got > 0 {
have_image, changed = true, true;
frame_time = shown;
lock(*r.mutex); r.stats.captured += 1; unlock(*r.mutex);
}
now = ticks_now();
if atomic_read(*r.paused) {
was_paused = true;
lock(*r.mutex); r.stats.paused_seconds = cast(float64) (atomic_read(*r.paused_ticks) + now - atomic_read(*r.pause_began)) / r.frequency; unlock(*r.mutex);
continue;
}
if was_paused {
was_paused = false;
changed = true;
turn = now;
frame_time = now; // the screen as it is, at the moment of resuming
lock(*r.mutex); r.stats.paused_seconds = cast(float64) atomic_read(*r.paused_ticks) / r.frequency; unlock(*r.mutex);
}
if !have_image continue;
repeat := !changed && now - last_emit >= r.frequency;
if !(changed && now >= turn - early) && !repeat continue;
turn = max(turn + period, now - period);
record_queue_frame(r, record_timeline(r, ifx repeat then now else frame_time));
last_emit = now;
changed = false;
lock(*r.mutex);
r.stats.emitted += 1;
if repeat r.stats.repeats += 1;
unlock(*r.mutex);
}
atomic_swap(*r.capture_ended, true);
signal(*r.ready);
return 0;
}
// The newest frame into a free buffer, queued for the encoder; the oldest waiting dropped when every
// buffer is taken.
record_queue_frame :: (r: *Recorder, ticks: s64) {
lock(*r.mutex);
if !r.free_count {
r.free[r.free_count] = r.queue[0];
r.free_count += 1;
for 1..r.queue_count - 1 r.queue[it - 1] = r.queue[it];
r.queue_count -= 1;
r.stats.dropped += 1;
}
r.free_count -= 1;
index := r.free[r.free_count];
unlock(*r.mutex);
frame := *r.frames[index];
memcpy(frame.pixels.data, r.latest.data, frame.pixels.count);
frame.ticks = ticks;
lock(*r.mutex);
r.queue[r.queue_count] = index;
r.queue_count += 1;
memcpy(r.last_frame.data, frame.pixels.data, frame.pixels.count);
unlock(*r.mutex);
signal(*r.ready);
}
record_encoder_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
while true {
wait_for(*r.ready, 100);
lock(*r.mutex);
index: s32 = -1;
if r.queue_count {
index = r.queue[0];
for 1..r.queue_count - 1 r.queue[it - 1] = r.queue[it];
r.queue_count -= 1;
}
unlock(*r.mutex);
if index < 0 {
if atomic_read(*r.capture_ended) break;
continue;
}
frame := *r.frames[index];
pts := max(frame.ticks, 0) * 90000 / r.frequency; // on the timeline already
ok, error := encoder_write(*r.encoder, frame.pixels.data, pts);
lock(*r.mutex);
r.free[r.free_count] = index;
r.free_count += 1;
if ok r.stats.written += 1;
r.stats.bytes = r.encoder.bytes;
unlock(*r.mutex);
if !ok {
record_fail(r, error);
while !atomic_read(*r.capture_ended) {
wait_for(*r.ready, 100);
lock(*r.mutex);
for 0..r.queue_count - 1 { r.free[r.free_count] = r.queue[it]; r.free_count += 1; }
r.queue_count = 0;
unlock(*r.mutex);
}
break;
}
}
return 0;
}
// Every 10 ms: the sources turned on or off as asked, their packets into the mix, and what every
// source has covered encoded.
record_audio_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
if !r.encoder.audio return 0;
mix: Audio_Mix;
mix.frame_size = r.encoder.audio_frame_size;
defer array_free(mix.samples);
MAX_FRAMES :: 4096;
samples := NewArray(MAX_FRAMES * AUDIO_CHANNELS, float);
defer array_free(samples);
error_text: [512] u8;
while true {
stopping := atomic_read(*r.stopping);
// The stream's sources as the toolbar wants them.
want := r.audio_on;
for * want it.* = atomic_read(it);
if !stopping && (want[0] != r.audio_sources[0].open || want[1] != r.audio_sources[1].open) {
if want[1] && !r.audio_sources[1].open && !mcap_sck_microphone_supported() {
atomic_swap(*r.audio_on[Audio_Source_Kind.MICROPHONE], false);
lock(*r.mutex); free(r.stats.audio_errors); r.stats.audio_errors = copy_string("microphone: it needs macOS 15 or later"); unlock(*r.mutex);
atomic_swap(*r.audio_failed[Audio_Source_Kind.MICROPHONE], true);
want[1] = false;
}
if mcap_sck_stream_set_audio(r.stream, xx ifx want[0] then 1 else 0, xx ifx want[1] then 1 else 0, error_text.data, error_text.count) {
for 0..1 {
if want[it] && !r.audio_sources[it].open mix.started[it] = false;
r.audio_sources[it].open = want[it];
}
record_note_audio(r);
log("[record] Sound now: %.", ifx r.stats.audio then r.stats.audio else "none");
} else {
why := to_string(error_text.data);
log_error("[record] The sound could not be changed: %", why);
for 0..1 if want[it] && !r.audio_sources[it].open {
atomic_swap(*r.audio_on[it], false);
lock(*r.mutex); free(r.stats.audio_errors); r.stats.audio_errors = sprint("% could not be turned on: %", r.audio_sources[it].name, why); unlock(*r.mutex);
atomic_swap(*r.audio_failed[it], true);
}
}
}
for * source, index: r.audio_sources {
if !source.open continue;
while true {
shown: s64;
frames := mcap_sck_next_audio(r.stream, xx index, samples.data, MAX_FRAMES, *shown);
if frames <= 0 break;
if atomic_read(*r.paused) continue;
packet := Audio_Packet.{samples = array_view(samples, 0, frames * AUDIO_CHANNELS), frames = frames, ticks = shown};
audio_mix_add(*mix, r, index, packet);
}
}
ok, error := audio_mix_encode(*mix, r, final = stopping);
if !ok { record_fail(r, error); break; }
if stopping break;
reset_temporary_storage();
sleep_milliseconds(10);
}
return 0;
}
record_fail :: (r: *Recorder, error: string) {
log_error("[record] %", error);
lock(*r.mutex);
if !r.error r.error = copy_string(error);
unlock(*r.mutex);
atomic_swap(*r.stopping, true);
}
#scope_module
// The shim (macos/capture.h).
Mcap_Sck_Stream :: struct { _: u8; }
mcap_capture :: #library,no_dll "macos/capture";
mcap_sck_stream_start :: (display_id: u32, x: float64, y: float64, w: float64, h: float64, width: s32, height: s32, fps: s32, show_cursor: s32,
system_audio: s32, microphone: s32, excluded: *u32, excluded_count: s32, error: *u8, error_size: s32) -> *Mcap_Sck_Stream #foreign mcap_capture;
mcap_sck_next_frame :: (s: *Mcap_Sck_Stream, pixels: *u8, host_time_ns: *s64, timeout_ms: s32, error: *u8, error_size: s32) -> s32 #foreign mcap_capture;
mcap_sck_next_audio :: (s: *Mcap_Sck_Stream, source: s32, samples: *float, max_frames: s32, host_time_ns: *s64) -> s32 #foreign mcap_capture;
mcap_sck_stream_set_audio :: (s: *Mcap_Sck_Stream, system_audio: s32, microphone: s32, error: *u8, error_size: s32) -> s32 #foreign mcap_capture;
mcap_sck_microphone_supported :: () -> s32 #foreign mcap_capture;
mcap_sck_stream_stop :: (s: *Mcap_Sck_Stream) #foreign mcap_capture;
ScreenCaptureKit_Framework :: #library,system,link_always "ScreenCaptureKit";
CoreMedia_Framework :: #library,system,link_always "CoreMedia";
CoreVideo_Framework :: #library,system,link_always "CoreVideo";
clock_gettime_nsec_np :: (clock: s32) -> u64 #foreign libc_for_clock;
libc_for_clock :: #library,system "libc";

36
modules/Screen_Record/macos/capture.h
View File

@ -15,3 +15,39 @@ typedef struct {
int mcap_sck_screenshots(Mcap_Sck_Shot *shots, int count, char *error, int error_size); int mcap_sck_screenshots(Mcap_Sck_Shot *shots, int count, char *error, int error_size);
void mcap_sck_free(void *pointer); void mcap_sck_free(void *pointer);
// ---------------------------------------------------------------- recording (macos-plan, phase 7)
// A stream of one display's region, with the sound wanted. Frames are kept one at a time, the
// newest; sound is queued, per source, until taken.
typedef struct Mcap_Sck_Stream Mcap_Sck_Stream;
enum { MCAP_SCK_SYSTEM = 0, MCAP_SCK_MICROPHONE = 1 };
// The region (x, y, w, h) in the display's points from its top left; width and height the frames'
// size in pixels. excluded: window numbers kept out of the frames (mcap's border and toolbar).
// The microphone comes from the stream itself, which macOS 15 has; on 14 it cannot be turned on.
// NULL, with why in error, when it could not start.
Mcap_Sck_Stream *mcap_sck_stream_start(uint32_t display_id, double x, double y, double w, double h,
int32_t width, int32_t height, int32_t fps, int show_cursor,
int system_audio, int microphone,
const uint32_t *excluded, int excluded_count,
char *error, int error_size);
// The newest frame since the last taken, as packed BGRA into pixels (width * height * 4), with the
// host time it was shown at (mach_absolute_time, in nanoseconds). 1 for a frame; 0 when none came in
// timeout_ms; -1 when the stream stopped by itself (error says why).
int mcap_sck_next_frame(Mcap_Sck_Stream *s, uint8_t *pixels, int64_t *host_time_ns, int timeout_ms, char *error, int error_size);
// The oldest sound packet of a source not yet taken: up to max_frames frames of 48 kHz stereo float,
// interleaved, into samples, and the host time of its first frame. Its frame count; 0 for none.
int mcap_sck_next_audio(Mcap_Sck_Stream *s, int source, float *samples, int max_frames, int64_t *host_time_ns);
// Turns the sources on and off while it runs (the stream's configuration changed). 1 when done.
int mcap_sck_stream_set_audio(Mcap_Sck_Stream *s, int system_audio, int microphone, char *error, int error_size);
// Whether this macOS can take the microphone from the stream.
int mcap_sck_microphone_supported(void);
// Stops it, waits for it, and frees it.
void mcap_sck_stream_stop(Mcap_Sck_Stream *s);

287
modules/Screen_Record/macos/capture.m
View File

@ -1,10 +1,13 @@
// ScreenCaptureKit for mcap on macOS, behind a plain C interface (capture.h): ScreenCaptureKit is // ScreenCaptureKit for mcap on macOS, behind a plain C interface (capture.h): ScreenCaptureKit is
// asynchronous, with completion blocks and its own queues, which is simpler to wait on here than from // asynchronous, with completion blocks and its own queues, which is simpler to wait on here than from
// Jai. first.jai compiles this into capture.a beside it. Recording (macos-plan, phase 7) comes here too. // Jai. first.jai compiles this into capture.a beside it. Stills for the screen capture overlay, and
// a stream for recording (modules/Screen_Record/macos.jai): its frames and sound, on host time.
#import "capture.h" #import "capture.h"
#import <ScreenCaptureKit/ScreenCaptureKit.h> #import <ScreenCaptureKit/ScreenCaptureKit.h>
#import <CoreGraphics/CoreGraphics.h> #import <CoreGraphics/CoreGraphics.h>
#import <CoreMedia/CoreMedia.h>
#import <CoreVideo/CoreVideo.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
@ -105,3 +108,285 @@ int mcap_sck_screenshots(Mcap_Sck_Shot *shots, int count, char *error, int error
void mcap_sck_free(void *pointer) { void mcap_sck_free(void *pointer) {
free(pointer); free(pointer);
} }
// ---------------------------------------------------------------- recording
#define AUDIO_RATE 48000
// Whether this is macOS 15 or later, for the microphone in the stream. Asked of NSProcessInfo rather
// than with @available, whose check comes from clang's runtime library, which Jai's link leaves out;
// the warnings for the calls it guards are off below.
static BOOL macos_15(void) {
static int known = -1;
if (known < 0) known = [[NSProcessInfo processInfo] isOperatingSystemAtLeastVersion:(NSOperatingSystemVersion) {15, 0, 0}] ? 1 : 0;
return known == 1;
}
#pragma clang diagnostic ignored "-Wunguarded-availability-new"
// A source's packets, waiting to be taken: interleaved stereo float at AUDIO_RATE.
@interface McapAudioPacket : NSObject
@property (nonatomic) NSData *samples;
@property (nonatomic) int64_t hostTime;
@end
@implementation McapAudioPacket
@end
@interface McapStream : NSObject <SCStreamOutput, SCStreamDelegate>
@property (nonatomic) SCStream *stream;
@property (nonatomic) SCStreamConfiguration *configuration;
@property (nonatomic) dispatch_queue_t queue;
@property (nonatomic) NSCondition *condition; // over everything below
@property (nonatomic) NSMutableData *frame; // the newest frame, packed BGRA
@property (nonatomic) int64_t frameTime;
@property (nonatomic) BOOL fresh; // a frame not yet taken
@property (nonatomic) int32_t width, height;
@property (nonatomic) NSString *failure; // the stream stopped by itself
@property (nonatomic) NSMutableArray<McapAudioPacket *> *system, *microphone;
@end
static int64_t host_nanoseconds(CMTime time) {
return (int64_t) (CMTimeGetSeconds(time) * 1e9);
}
@implementation McapStream
- (void)stream:(SCStream *)stream didOutputSampleBuffer:(CMSampleBufferRef)buffer ofType:(SCStreamOutputType)type {
if (!CMSampleBufferIsValid(buffer)) return;
if (type == SCStreamOutputTypeScreen) [self takeFrame:buffer];
else if (type == SCStreamOutputTypeAudio) [self takeAudio:buffer into:self.system];
else if (macos_15()) { if (type == SCStreamOutputTypeMicrophone) [self takeAudio:buffer into:self.microphone]; }
}
- (void)stream:(SCStream *)stream didStopWithError:(NSError *)error {
[self.condition lock];
self.failure = error.localizedDescription ?: @"the screen stopped being recorded";
[self.condition broadcast];
[self.condition unlock];
}
// A complete frame (not an idle one, which repeats the last), packed into self.frame.
- (void)takeFrame:(CMSampleBufferRef)buffer {
CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(buffer, false);
if (attachments && CFArrayGetCount(attachments) > 0) {
NSDictionary *info = (__bridge NSDictionary *) CFArrayGetValueAtIndex(attachments, 0);
NSNumber *status = info[SCStreamFrameInfoStatus];
if (status && status.integerValue != SCFrameStatusComplete) return;
}
CVPixelBufferRef image = CMSampleBufferGetImageBuffer(buffer);
if (!image) return;
CVPixelBufferLockBaseAddress(image, kCVPixelBufferLock_ReadOnly);
const uint8_t *base = CVPixelBufferGetBaseAddress(image);
size_t stride = CVPixelBufferGetBytesPerRow(image);
size_t w = MIN((size_t) self.width, CVPixelBufferGetWidth(image));
size_t h = MIN((size_t) self.height, CVPixelBufferGetHeight(image));
[self.condition lock];
uint8_t *out = self.frame.mutableBytes;
size_t row = (size_t) self.width * 4;
if (base) for (size_t y = 0; y < h; y++) memcpy(out + y * row, base + y * stride, w * 4);
self.frameTime = host_nanoseconds(CMSampleBufferGetPresentationTimeStamp(buffer));
self.fresh = YES;
[self.condition signal];
[self.condition unlock];
CVPixelBufferUnlockBaseAddress(image, kCVPixelBufferLock_ReadOnly);
}
// A sound buffer as interleaved stereo float at AUDIO_RATE, onto the source's queue.
- (void)takeAudio:(CMSampleBufferRef)buffer into:(NSMutableArray<McapAudioPacket *> *)queue {
CMFormatDescriptionRef format = CMSampleBufferGetFormatDescription(buffer);
const AudioStreamBasicDescription *asbd = format ? CMAudioFormatDescriptionGetStreamBasicDescription(format) : NULL;
if (!asbd || !(asbd->mFormatFlags & kAudioFormatFlagIsFloat) || asbd->mBitsPerChannel != 32) return;
CMItemCount frames = CMSampleBufferGetNumSamples(buffer);
if (frames <= 0) return;
size_t needed = 0;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(buffer, &needed, NULL, 0, NULL, NULL, 0, NULL);
AudioBufferList *list = malloc(needed);
CMBlockBufferRef block = NULL;
OSStatus status = CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(buffer, NULL, list, needed, NULL, NULL,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment, &block);
if (status != noErr) { free(list); return; }
int channels = (int) asbd->mChannelsPerFrame;
BOOL planar = (asbd->mFormatFlags & kAudioFormatFlagIsNonInterleaved) != 0;
// Each frame's left and right: a mono source on both.
float (^sample)(CMItemCount, int) = ^float(CMItemCount i, int c) {
int channel = channels >= 2 ? c : 0;
if (planar) return ((float *) list->mBuffers[channel < (int) list->mNumberBuffers ? channel : 0].mData)[i];
return ((float *) list->mBuffers[0].mData)[i * channels + channel];
};
double ratio = asbd->mSampleRate > 0 ? AUDIO_RATE / asbd->mSampleRate : 1;
CMItemCount out_frames = (CMItemCount) (frames * ratio + 0.5);
NSMutableData *samples = [NSMutableData dataWithLength:(size_t) out_frames * 2 * sizeof(float)];
float *out = samples.mutableBytes;
for (CMItemCount i = 0; i < out_frames; i++) {
// Linearly between the source's frames, when its rate is not AUDIO_RATE.
double at = ratio == 1 ? (double) i : i / ratio;
CMItemCount a = (CMItemCount) at;
CMItemCount b = MIN(a + 1, frames - 1);
float t = (float) (at - (double) a);
for (int c = 0; c < 2; c++) out[i * 2 + c] = sample(MIN(a, frames - 1), c) * (1 - t) + sample(b, c) * t;
}
CFRelease(block);
free(list);
McapAudioPacket *packet = [McapAudioPacket new];
packet.samples = samples;
packet.hostTime = host_nanoseconds(CMSampleBufferGetPresentationTimeStamp(buffer));
[self.condition lock];
[queue addObject:packet];
[self.condition unlock];
}
@end
static void configure_audio(SCStreamConfiguration *configuration, int system_audio, int microphone) {
configuration.capturesAudio = system_audio ? YES : NO;
configuration.sampleRate = AUDIO_RATE;
configuration.channelCount = 2;
configuration.excludesCurrentProcessAudio = YES;
if (macos_15()) configuration.captureMicrophone = microphone ? YES : NO;
}
int mcap_sck_microphone_supported(void) {
if (macos_15()) return 1;
return 0;
}
Mcap_Sck_Stream *mcap_sck_stream_start(uint32_t display_id, double x, double y, double w, double h,
int32_t width, int32_t height, int32_t fps, int show_cursor,
int system_audio, int microphone,
const uint32_t *excluded, int excluded_count,
char *error, int error_size) {
@autoreleasepool {
__block SCShareableContent *content = nil;
__block NSError *content_error = nil;
dispatch_semaphore_t got = dispatch_semaphore_create(0);
[SCShareableContent getShareableContentExcludingDesktopWindows:NO onScreenWindowsOnly:NO
completionHandler:^(SCShareableContent *c, NSError *e) {
content = c;
content_error = e;
dispatch_semaphore_signal(got);
}];
if (dispatch_semaphore_wait(got, dispatch_time(DISPATCH_TIME_NOW, 5 * NSEC_PER_SEC)) != 0 || !content) {
set_error(error, error_size, content_error ? content_error.localizedDescription : @"ScreenCaptureKit did not list the screens");
return NULL;
}
SCDisplay *display = nil;
for (SCDisplay *d in content.displays) if (d.displayID == display_id) display = d;
if (!display) { set_error(error, error_size, [NSString stringWithFormat:@"ScreenCaptureKit has no display %u", display_id]); return NULL; }
NSMutableArray<SCWindow *> *leave_out = [NSMutableArray array];
for (SCWindow *window in content.windows)
for (int i = 0; i < excluded_count; i++) if (window.windowID == excluded[i]) [leave_out addObject:window];
SCContentFilter *filter = [[SCContentFilter alloc] initWithDisplay:display excludingWindows:leave_out];
SCStreamConfiguration *configuration = [SCStreamConfiguration new];
configuration.sourceRect = CGRectMake(x, y, w, h);
configuration.width = (size_t) width;
configuration.height = (size_t) height;
configuration.minimumFrameInterval = CMTimeMake(1, fps > 0 ? fps : 30);
configuration.pixelFormat = kCVPixelFormatType_32BGRA;
configuration.colorSpaceName = kCGColorSpaceSRGB;
configuration.showsCursor = show_cursor ? YES : NO;
configuration.queueDepth = 6;
configure_audio(configuration, system_audio, microphone);
McapStream *holder = [McapStream new];
holder.configuration = configuration;
holder.condition = [NSCondition new];
holder.width = width;
holder.height = height;
holder.frame = [NSMutableData dataWithLength:(size_t) width * (size_t) height * 4];
holder.system = [NSMutableArray array];
holder.microphone = [NSMutableArray array];
holder.queue = dispatch_queue_create("mcap.record", DISPATCH_QUEUE_SERIAL);
holder.stream = [[SCStream alloc] initWithFilter:filter configuration:configuration delegate:holder];
NSError *add_error = nil;
BOOL added = [holder.stream addStreamOutput:holder type:SCStreamOutputTypeScreen sampleHandlerQueue:holder.queue error:&add_error];
if (added) added = [holder.stream addStreamOutput:holder type:SCStreamOutputTypeAudio sampleHandlerQueue:holder.queue error:&add_error];
if (added) if (macos_15()) added = [holder.stream addStreamOutput:holder type:SCStreamOutputTypeMicrophone sampleHandlerQueue:holder.queue error:&add_error];
if (!added) { set_error(error, error_size, add_error.localizedDescription ?: @"ScreenCaptureKit would not take the stream's outputs"); return NULL; }
__block NSError *start_error = nil;
dispatch_semaphore_t started = dispatch_semaphore_create(0);
[holder.stream startCaptureWithCompletionHandler:^(NSError *e) { start_error = e; dispatch_semaphore_signal(started); }];
if (dispatch_semaphore_wait(started, dispatch_time(DISPATCH_TIME_NOW, 10 * NSEC_PER_SEC)) != 0) {
set_error(error, error_size, @"ScreenCaptureKit did not start recording in time");
return NULL;
}
if (start_error) { set_error(error, error_size, start_error.localizedDescription); return NULL; }
return (__bridge_retained Mcap_Sck_Stream *) holder;
}
}
int mcap_sck_next_frame(Mcap_Sck_Stream *s, uint8_t *pixels, int64_t *host_time_ns, int timeout_ms, char *error, int error_size) {
McapStream *holder = (__bridge McapStream *) s;
[holder.condition lock];
NSDate *until = [NSDate dateWithTimeIntervalSinceNow:timeout_ms / 1000.0];
while (!holder.fresh && !holder.failure) if (![holder.condition waitUntilDate:until]) break;
int result = 0;
if (holder.failure) {
set_error(error, error_size, holder.failure);
result = -1;
} else if (holder.fresh) {
memcpy(pixels, holder.frame.bytes, holder.frame.length);
*host_time_ns = holder.frameTime;
holder.fresh = NO;
result = 1;
}
[holder.condition unlock];
return result;
}
int mcap_sck_next_audio(Mcap_Sck_Stream *s, int source, float *samples, int max_frames, int64_t *host_time_ns) {
McapStream *holder = (__bridge McapStream *) s;
NSMutableArray<McapAudioPacket *> *queue = source == MCAP_SCK_MICROPHONE ? holder.microphone : holder.system;
[holder.condition lock];
int frames = 0;
if (queue.count) {
McapAudioPacket *packet = queue[0];
int available = (int) (packet.samples.length / (2 * sizeof(float)));
frames = MIN(available, max_frames);
memcpy(samples, packet.samples.bytes, (size_t) frames * 2 * sizeof(float));
*host_time_ns = packet.hostTime;
if (frames == available) [queue removeObjectAtIndex:0];
else { // the rest stays, its time moved on
packet.samples = [packet.samples subdataWithRange:NSMakeRange((size_t) frames * 2 * sizeof(float), packet.samples.length - (size_t) frames * 2 * sizeof(float))];
packet.hostTime += (int64_t) frames * 1000000000LL / AUDIO_RATE;
}
}
[holder.condition unlock];
return frames;
}
int mcap_sck_stream_set_audio(Mcap_Sck_Stream *s, int system_audio, int microphone, char *error, int error_size) {
@autoreleasepool {
McapStream *holder = (__bridge McapStream *) s;
if (microphone && !mcap_sck_microphone_supported()) { set_error(error, error_size, @"the microphone needs macOS 15 or later"); return 0; }
configure_audio(holder.configuration, system_audio, microphone);
__block NSError *update_error = nil;
dispatch_semaphore_t done = dispatch_semaphore_create(0);
[holder.stream updateConfiguration:holder.configuration completionHandler:^(NSError *e) { update_error = e; dispatch_semaphore_signal(done); }];
if (dispatch_semaphore_wait(done, dispatch_time(DISPATCH_TIME_NOW, 5 * NSEC_PER_SEC)) != 0) {
set_error(error, error_size, @"ScreenCaptureKit did not change the sound in time");
return 0;
}
if (update_error) { set_error(error, error_size, update_error.localizedDescription); return 0; }
// A source turned off keeps nothing queued.
[holder.condition lock];
if (!system_audio) [holder.system removeAllObjects];
if (!microphone) [holder.microphone removeAllObjects];
[holder.condition unlock];
return 1;
}
}
void mcap_sck_stream_stop(Mcap_Sck_Stream *s) {
if (!s) return;
McapStream *holder = (__bridge_transfer McapStream *) s;
@autoreleasepool {
dispatch_semaphore_t stopped = dispatch_semaphore_create(0);
[holder.stream stopCaptureWithCompletionHandler:^(NSError *e) { dispatch_semaphore_signal(stopped); }];
dispatch_semaphore_wait(stopped, dispatch_time(DISPATCH_TIME_NOW, 5 * NSEC_PER_SEC));
[holder.stream removeStreamOutput:holder type:SCStreamOutputTypeScreen error:nil];
[holder.stream removeStreamOutput:holder type:SCStreamOutputTypeAudio error:nil];
if (macos_15()) [holder.stream removeStreamOutput:holder type:SCStreamOutputTypeMicrophone error:nil];
dispatch_sync(holder.queue, ^{}); // the last sample handler is done
}
}

772
modules/Screen_Record/module.jai
View File

@ -1,33 +1,14 @@
// Screen_Record: a region of one screen recorded to H.264 in an MP4, for the mcap app's screen // Screen_Record: a region of one screen recorded to H.264 in an MP4, for the mcap app's screen
// recording and tools/record_probe.jai (video-capture-plan.html). Windows only. // recording and tools/record_probe.jai (video-capture-plan.html, macos-plan phase 7). On Windows with
// // Desktop Duplication or Windows.Graphics.Capture and WASAPI (windows.jai); on macOS with
// Two threads. The capture thread takes the screen with DXGI Desktop Duplication (dxgi_source.jai), // ScreenCaptureKit, its frames and sound alike, through modules/Screen_Record/macos/capture.m
// or with Windows.Graphics.Capture (wgc_source.jai) where duplication cannot run, each on a D3D11 // (macos.jai). Either way to FFmpeg (encoder.jai), the sound mixed onto the video's timeline
// device of its own; copies the region on the GPU; reads it back through a ring of staging textures, // (audio_mix.jai), and made a regular MP4 at the end (remux.jai).
// so Map never waits on the GPU (capture_gpu.jai); and, for duplication, draws the cursor in
// (cursor.jai). The encoder thread hands the frames to FFmpeg (encoder.jai). Between them is a pool of
// RECORD_BUFFERS frames; when the encoder falls RECORD_QUEUE frames behind, the oldest waiting is
// dropped and counted.
//
// A frame is taken at most fps times a second, and only when the screen or the cursor changed, with
// one a second while nothing does, so players can seek. Each carries the time it was taken, less the
// time spent paused (record_pause), so a pause leaves no gap in the video.
//
// Sound (system_audio, microphone) has a thread of its own (audio_source.jai's WASAPI): each packet is
// put on the same timeline as the frames, by when it was heard, less the same pauses, into one mix,
// and an AAC frame of it is encoded once every source has had its say up to there. A source that sends
// nothing (the computer playing nothing) is silence. Either source can be turned on and off while
// recording (record_set_audio); with audio_track, the MP4 has a sound track even when no source is on
// at the start, so one can be.
// //
// recorder: Recorder; // recorder: Recorder;
// ok, error, fall_back := record_start(*recorder, .{path = "out.mp4", screen = 0}); // ok, error := record_start(*recorder, .{path = "out.mp4", ...});
// ... // ...
// ok, error = record_stop(*recorder); // ok, error = record_stop(*recorder);
//
// With capture .AUTO (the default), Desktop Duplication is tried first and Windows.Graphics.Capture
// takes over when it cannot run: on a laptop with two GPUs, on a rotated screen, or with too many
// duplications of the screen running. fall_back is only returned for .DXGI, which does not fall back.
RECORD_MIN_WIDTH :: 240; // a smaller region is refused RECORD_MIN_WIDTH :: 240; // a smaller region is refused
RECORD_MIN_HEIGHT :: 180; RECORD_MIN_HEIGHT :: 180;
@ -40,750 +21,15 @@ record_size_allowed :: (width: s64, height: s64) -> bool {
return width >= RECORD_MIN_WIDTH && height >= RECORD_MIN_HEIGHT; return width >= RECORD_MIN_WIDTH && height >= RECORD_MIN_HEIGHT;
} }
Record_Screen :: struct { #if OS == .WINDOWS #load "windows.jai";
adapter_index, output_index: u32; #if OS == .MACOS #load "macos.jai";
adapter: string; // "NVIDIA GeForce RTX 5080 Laptop GPU" #load "audio_mix.jai";
name: string; // "\\.\DISPLAY1"
desktop: RECT; // on the virtual desktop
rotation: DXGI_MODE_ROTATION;
monitor: HMONITOR;
}
Record_Capture :: enum u8 {
AUTO; // Desktop Duplication, else Windows.Graphics.Capture
DXGI; // Desktop Duplication only
WGC; // Windows.Graphics.Capture only
}
// For record-probe: paths a machine may not take by itself.
Record_Testing :: enum_flags u8 {
DXGI_UNSUPPORTED; // Desktop Duplication fails as on a laptop with two GPUs (DXGI_ERROR_UNSUPPORTED)
WGC_POLL; // Windows.Graphics.Capture polled, as when it will not take the FrameArrived handler
KEEP_BORDER; // Windows.Graphics.Capture's yellow border left on
LOSE_DUPLICATION; // Desktop Duplication lost every 3 s, as a mode change or the secure desktop loses it
}
Record_Options :: struct {
path: string; // the .mp4 written
screen: s32 = -1; // into record_list_screens(); -1: the one holding region
region: RECT; // on the virtual desktop; empty: the whole screen
fps: s32 = 30;
cursor := true;
encoder: string; // "" tries RECORD_ENCODERS in turn
capture: Record_Capture;
legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe
testing: Record_Testing;
system_audio: bool; // what the computer plays (the default output device)
microphone: bool; // the default input device; both at the start: record_set_audio turns them on and off
audio_track: bool; // a sound track whatever is on at the start, so a source can be turned on later
}
Record_Stats :: struct {
source: string; // "DXGI DuplicateOutput1 on <adapter>", "Windows.Graphics.Capture on <adapter>, border off"
fell_back_from: string; // why Desktop Duplication could not run, when it fell back
encoder: string;
width, height: s32;
captured: s64; // new images of the screen
pointer_moves: s64; // cursor changes with no new image (Desktop Duplication)
emitted: s64; // frames sent to be encoded
repeats: s64; // of them, the once-a-second repeat of an unchanged screen
written: s64; // encoded
dropped: s64; // the encoder was too far behind
lost: s64; // times the duplication was lost and made again
paused_seconds: float64; // left out of the video
audio: string; // the sound recorded: "Speakers (Realtek(R) Audio); Microphone (USB Audio)", or ""
audio_frames: s64; // AAC frames encoded
audio_errors: string; // sources that could not be opened, or were lost
copy_ticks, readback_ticks, encode_ticks: s64; // totals, in QueryPerformanceCounter ticks
bytes: s64; // encoded
seconds: float64; // from start to stop
}
Recorder :: struct {
options: Record_Options;
screen: Record_Screen;
local: RECT; // the region, in the screen's pixels
dxgi: Dxgi_Source; // the capture thread's: one of these is open
wgc: Wgc_Source;
using_wgc: bool;
encoder: Encoder;
width, height: s32;
frequency: s64; // QueryPerformanceCounter's
start_ticks: s64;
capture_thread, encoder_thread: Thread;
capture_started, encoder_started, audio_started: bool;
audio_thread: Thread;
audio_opened: Semaphore; // the audio thread has opened its sources
audio_go: Semaphore; // the encoder is open: the audio thread may write
audio_sources: [2] Audio_Source;
audio_open: bool; // any source opened, or audio_track: the MP4 has sound
audio_on: [2] bool; // each source is wanted (by Audio_Source_Kind): record_set_audio; cleared when it cannot be opened
audio_failed: [2] bool; // it could not be opened after being turned on: record_audio_failed says once
zero_ticks: s64; // the recording's time 0, for frames and sound alike
paused_ticks: s64; // time paused so far, for the capture and audio threads; set by record_pause
pause_began: s64;
running: bool;
stopping: bool; // set by record_stop; the capture thread ends, then the encoder
paused: bool; // set by record_pause; the capture thread takes nothing while it is
capture_ended: bool;
opened: Semaphore; // the capture thread has opened its source, or failed to
open_ok, open_fall_back: bool;
open_error: string;
mutex: Mutex;
ready: Semaphore; // signalled for each frame queued, and once when capture ends
frames: [RECORD_BUFFERS] Record_Frame;
free: [RECORD_BUFFERS] s32;
free_count: s32;
queue: [RECORD_BUFFERS] s32; // oldest first
queue_count: s32;
last_frame: [] u8; // the last frame queued, BGRA, for record_last_frame
stats: Record_Stats;
error: string; // why it stopped by itself; heap
}
Record_Frame :: struct {
pixels: [] u8; // BGRA, rows packed
ticks: s64;
}
// Opens the screen, the encoder and the file, and starts recording.
record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: string, fall_back: bool {
r.* = .{};
r.options = options;
r.options.path = copy_string(options.path);
r.options.encoder = copy_string(options.encoder);
if r.options.fps <= 0 r.options.fps = 30;
screens := record_list_screens();
if !screens.count return false, "no screen to record", false;
screen_index := options.screen;
region := options.region;
has_region := region.right > region.left && region.bottom > region.top;
if screen_index < 0 {
if !has_region screen_index = 0;
else for screens if rect_inside(region, it.desktop) { screen_index = xx it_index; break; }
if screen_index < 0 return false, "the region is not on one screen", false;
}
if screen_index >= screens.count return false, tprint("there is no screen %", screen_index), false;
r.screen = screens[screen_index];
r.screen.adapter = copy_string(r.screen.adapter);
r.screen.name = copy_string(r.screen.name);
if !has_region region = r.screen.desktop;
if !rect_inside(region, r.screen.desktop) return false, "the region is not on that screen", false;
// The minimum first, then even both ways, in the screen's pixels: H.264's 4:2:0 colour needs it.
// RECORD_MIN_WIDTH and _HEIGHT are even, so what passes stays at least that size.
#assert RECORD_MIN_WIDTH % 2 == 0 && RECORD_MIN_HEIGHT % 2 == 0;
if !record_size_allowed(region.right - region.left, region.bottom - region.top)
return false, tprint("% x % is smaller than % x %", region.right - region.left, region.bottom - region.top, RECORD_MIN_WIDTH, RECORD_MIN_HEIGHT), false;
d := r.screen.desktop;
r.local = .{region.left - d.left, region.top - d.top, region.right - d.left, region.bottom - d.top};
r.local.right = r.local.left + ((r.local.right - r.local.left) & cast(s32) ~1);
r.local.bottom = r.local.top + ((r.local.bottom - r.local.top) & cast(s32) ~1);
r.width = r.local.right - r.local.left;
r.height = r.local.bottom - r.local.top;
r.stats.width, r.stats.height = r.width, r.height;
init(*r.mutex, "record");
init(*r.ready);
init(*r.opened);
for * r.frames it.pixels = NewArray(cast(s64) r.width * r.height * 4, u8, initialized = false);
for 0..RECORD_BUFFERS - 1 r.free[it] = cast(s32) it;
r.free_count = RECORD_BUFFERS;
r.last_frame = NewArray(cast(s64) r.width * r.height * 4, u8);
frequency: LARGE_INTEGER;
QueryPerformanceFrequency(*frequency);
r.frequency = frequency.QuadPart;
r.start_ticks = ticks_now();
r.zero_ticks = r.start_ticks;
// The source is opened on the capture thread, which uses it: WinRT's objects belong to the
// apartment they are made in.
thread_init(*r.capture_thread, record_capture_proc, name = "record capture");
r.capture_started = true;
thread_start(*r.capture_thread, r);
wait_for(*r.opened);
if !r.open_ok {
error, fall_back := copy_string(r.open_error,, temp), r.open_fall_back;
record_free(r);
return false, error, fall_back;
}
// The sound's sources are opened on their own thread (COM, for WASAPI), before the encoder, which
// has a sound stream only when one opened.
r.audio_on = .[r.options.system_audio, r.options.microphone];
if r.options.system_audio || r.options.microphone || r.options.audio_track {
init(*r.audio_opened);
init(*r.audio_go);
thread_init(*r.audio_thread, record_audio_proc, name = "record audio");
r.audio_started = true;
thread_start(*r.audio_thread, r);
wait_for(*r.audio_opened);
}
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder, audio = r.audio_open);
if tried log("[record] Encoders that did not open: %", tried);
if !ok {
error = copy_string(error,, temp);
record_free(r);
return false, error, false;
}
r.stats.encoder = copy_string(r.encoder.name);
if r.audio_started signal(*r.audio_go);
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
r.encoder_started = true;
r.running = true;
thread_start(*r.encoder_thread, r);
return true, "", false;
}
// Stops, finishes the file and frees everything. ok is false when recording stopped by itself on an
// error, or the file could not be finished; what was recorded up to then is in the file all the same.
record_stop :: (r: *Recorder) -> ok: bool, error: string {
if !r.running return false, "not recording";
atomic_swap(*r.stopping, true);
thread_is_done(*r.capture_thread, -1);
thread_is_done(*r.encoder_thread, -1);
if r.audio_started thread_is_done(*r.audio_thread, -1);
r.running = false;
closed, close_error := encoder_close(*r.encoder);
r.stats.seconds = cast(float64) (ticks_now() - r.start_ticks) / r.frequency;
r.stats.bytes = r.encoder.bytes;
error := ifx r.error then copy_string(r.error,, temp) else "";
record_free(r);
if error return false, error;
if !closed return false, close_error;
return true, "";
}
// Pauses or resumes. While paused nothing is taken, and the time is left out of the video, sound and
// all: the time paused is kept here, once, for the capture and audio threads both.
record_pause :: (r: *Recorder, pause: bool) {
if !r.running || atomic_read(*r.paused) == pause return;
now := ticks_now();
if pause {
atomic_write(*r.pause_began, now);
atomic_swap(*r.paused, true);
} else {
atomic_add(*r.paused_ticks, now - atomic_read(*r.pause_began)); // before paused clears, so no thread sees it unpaused with the old total
atomic_swap(*r.paused, false);
}
}
// Turns a source of sound on or off while recording. It is opened only while on, so Windows shows the
// microphone in use only then. False when the recording has no sound track to put it in.
record_set_audio :: (r: *Recorder, kind: Audio_Source_Kind, on: bool) -> bool {
if !r.running || !r.audio_open return false;
atomic_swap(*r.audio_on[kind], on);
return true;
}
record_audio_on :: (r: *Recorder, kind: Audio_Source_Kind) -> bool {
return r.running && atomic_read(*r.audio_on[kind]);
}
// Whether a source could not be opened when turned on (it is off again then), and why; once.
record_audio_failed :: (r: *Recorder, kind: Audio_Source_Kind) -> failed: bool, error: string {
if !r.running || !atomic_read(*r.audio_failed[kind]) return false, "";
atomic_swap(*r.audio_failed[kind], false);
lock(*r.mutex);
defer unlock(*r.mutex);
return true, copy_string(r.stats.audio_errors,, temp);
}
// Ticks on the recording's timeline: time 0 at its start, less the time paused.
record_timeline :: (r: *Recorder, ticks: s64) -> s64 {
return ticks - r.zero_ticks - atomic_read(*r.paused_ticks);
}
record_paused :: (r: *Recorder) -> bool {
return r.running && atomic_read(*r.paused);
}
// As they are now; strings are the recorder's.
record_stats :: (r: *Recorder) -> Record_Stats {
if !r.running return r.stats;
lock(*r.mutex);
defer unlock(*r.mutex);
return r.stats;
}
// The last frame queued, BGRA (temp), cursor and all.
record_last_frame :: (r: *Recorder) -> pixels: [] u8, width: s32, height: s32 {
if r.running lock(*r.mutex);
pixels := array_copy(r.last_frame,, temp);
if r.running unlock(*r.mutex);
return pixels, r.width, r.height;
}
// Whether recording stopped by itself (the screen went away, the encoder failed); why, then.
record_failed :: (r: *Recorder) -> bool, string {
lock(*r.mutex);
defer unlock(*r.mutex);
return r.error.count > 0, r.error;
}
#scope_file
Staged :: struct { pending: bool; ticks: s64; pointer: POINT; pointer_visible: bool; }
// Stops the capture thread if it runs, and frees what record_start made. The stats' strings are kept.
record_free :: (r: *Recorder) {
atomic_swap(*r.stopping, true);
if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; }
if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; }
if r.audio_started {
signal(*r.audio_go); // in case it waits for an encoder that did not open
thread_is_done(*r.audio_thread, -1);
thread_deinit(*r.audio_thread);
destroy(*r.audio_opened);
destroy(*r.audio_go);
r.audio_started = false;
}
if r.encoder.format encoder_close(*r.encoder);
for * r.frames array_free(it.pixels);
array_free(r.last_frame);
r.last_frame = .[];
destroy(*r.opened);
destroy(*r.ready);
destroy(*r.mutex);
free(r.open_error);
free(r.error);
r.open_error, r.error = "", "";
}
// Desktop Duplication, unless asked for Windows.Graphics.Capture, which takes over when duplication
// cannot run here and the options allow.
record_open_source :: (r: *Recorder) -> bool {
if r.options.capture != .WGC {
ok, error, fall_back := dxgi_open(*r.dxgi, r.screen, r.local, r.options.legacy_duplicate);
if ok && r.options.testing & .DXGI_UNSUPPORTED {
ok, error, fall_back = false, "Desktop Duplication (testing): 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", true;
}
if ok {
r.stats.source = sprint("DXGI % on %", r.dxgi.method, r.screen.adapter);
return true;
}
dxgi_close(*r.dxgi);
if r.options.capture == .DXGI || !fall_back {
r.open_error, r.open_fall_back = copy_string(error), fall_back;
return false;
}
log("[record] Desktop Duplication cannot run here (%); Windows.Graphics.Capture instead.", error);
r.stats.fell_back_from = copy_string(error);
}
ok, error := wgc_open(*r.wgc, r.screen, r.local, r.options.cursor, r.options.testing);
if !ok {
wgc_close(*r.wgc);
r.open_error = copy_string(error);
return false;
}
r.using_wgc = true;
r.stats.source = sprint("Windows.Graphics.Capture on %, %%", r.screen.adapter, r.wgc.border, ifx r.wgc.polling then ", polled" else "");
return true;
}
record_capture_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
// Aware of each monitor's DPI, whatever the process is: Desktop Duplication refuses a thread that is
// not (DXGI_ERROR_UNSUPPORTED), and the mcap app's ImGui makes only its main thread aware.
gpu_thread_dpi_aware();
wgc_thread_start();
defer wgc_thread_end();
r.open_ok = record_open_source(r);
signal(*r.opened);
if !r.open_ok return 0;
gpu: *Capture_Gpu = ifx r.using_wgc then *r.wgc.gpu else *r.dxgi.gpu;
defer if r.using_wgc wgc_close(*r.wgc); else dxgi_close(*r.dxgi);
// A change is sent as soon as it comes once its turn is due: turns are a frame's time apart on a
// schedule that moves on a frame at a time, so the rate never passes fps, and a tenth of a frame
// early counts as on time, so a 60 Hz screen's frames, which jitter, all make a 60 fps cap.
period := r.frequency / r.options.fps;
early := period / 10;
turn := ticks_now();
last_emit: s64 = -r.frequency;
have_image, changed := false, false;
staged: [RECORD_STAGING] Staged;
emits: s64;
was_paused := false;
last_lost := ticks_now(); // for Record_Testing.LOSE_DUPLICATION
while !atomic_read(*r.stopping) {
now := ticks_now();
due := ifx changed then turn - early else last_emit + r.frequency; // the change, or the repeat
wait := cast(u32) clamp((due - now) * 1000 / r.frequency, 0, 100);
next: Capture_Next;
error: string;
if r.using_wgc next, error = wgc_next(*r.wgc, wait);
else next, error = dxgi_next(*r.dxgi, wait);
if r.options.testing & .LOSE_DUPLICATION && !r.using_wgc && ticks_now() - last_lost >= 3 * r.frequency {
next, last_lost = Capture_Next.LOST, ticks_now();
}
if next == {
case .FAILED;
record_fail(r, error);
break;
case .LOST;
// A mode change, the secure desktop, a game going fullscreen: duplicated again, and on.
lock(*r.mutex); r.stats.lost += 1; unlock(*r.mutex);
while !atomic_read(*r.stopping) {
hr := dxgi_duplicate(*r.dxgi);
if SUCCEEDED(hr) break;
if !hresult_is(hr, DXGI_ERROR_ACCESS_LOST) && !hresult_is(hr, 0x80070005) && !hresult_is(hr, DXGI_ERROR_NOT_CURRENTLY_AVAILABLE) {
record_fail(r, hresult_text("the screen could not be duplicated again", hr));
break;
}
sleep_milliseconds(250);
}
if atomic_read(*r.stopping) || r.error break;
continue;
case .IMAGE; #through;
case .IMAGE_AND_POINTER;
have_image, changed = true, true;
lock(*r.mutex);
r.stats.captured += 1;
r.stats.copy_ticks = gpu.copy_ticks;
unlock(*r.mutex);
case .POINTER;
if r.options.cursor && have_image {
changed = true;
lock(*r.mutex); r.stats.pointer_moves += 1; unlock(*r.mutex);
}
}
// Paused: the source is still drained, so it never stalls, but nothing is sent. On resuming, the
// screen as it is now is sent at once.
now = ticks_now();
if atomic_read(*r.paused) {
was_paused = true;
lock(*r.mutex); r.stats.paused_seconds = cast(float64) (atomic_read(*r.paused_ticks) + now - atomic_read(*r.pause_began)) / r.frequency; unlock(*r.mutex);
continue;
}
if was_paused {
was_paused = false;
changed = true;
turn = now;
lock(*r.mutex); r.stats.paused_seconds = cast(float64) atomic_read(*r.paused_ticks) / r.frequency; unlock(*r.mutex);
}
if !have_image continue;
repeat := !changed && now - last_emit >= r.frequency;
if !(changed && now >= turn - early) && !repeat continue;
turn = max(turn + period, now - period); // after a quiet spell, no burst to catch up
// Copied to staging now; read back two frames on, when the GPU is done with it.
slot := emits % RECORD_STAGING;
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
ID3D11DeviceContext_CopyResource(gpu.device_context, gpu.staging[slot], gpu.latest);
staged[slot] = .{pending = true, ticks = record_timeline(r, now)};
if !r.using_wgc staged[slot].pointer, staged[slot].pointer_visible = r.dxgi.pointer_position, r.dxgi.pointer_visible;
emits += 1;
last_emit = now;
changed = false;
lock(*r.mutex);
r.stats.emitted += 1;
if repeat r.stats.repeats += 1;
unlock(*r.mutex);
}
// What is still staged, oldest first.
for 0..RECORD_STAGING - 1 {
slot := (emits + it) % RECORD_STAGING;
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
}
atomic_swap(*r.capture_ended, true);
signal(*r.ready);
return 0;
}
// Copies a staging texture into a free frame, draws the cursor (Desktop Duplication leaves it out)
// and queues it.
record_read_back :: (r: *Recorder, gpu: *Capture_Gpu, slot: s64, staged: Staged) {
t := ticks_now();
lock(*r.mutex);
if !r.free_count { // every frame is waiting: drop the oldest
r.free[r.free_count] = r.queue[0];
r.free_count += 1;
for 1..r.queue_count - 1 r.queue[it - 1] = r.queue[it];
r.queue_count -= 1;
r.stats.dropped += 1;
}
r.free_count -= 1;
index := r.free[r.free_count];
unlock(*r.mutex);
frame := *r.frames[index];
mapped: D3D11_MAPPED_SUBRESOURCE;
hr := ID3D11DeviceContext_Map(gpu.device_context, gpu.staging[slot], 0, .READ, 0, *mapped);
if FAILED(hr) {
lock(*r.mutex); r.free[r.free_count] = index; r.free_count += 1; unlock(*r.mutex);
record_fail(r, hresult_text("reading the frame back", hr));
return;
}
row := cast(s64) r.width * 4;
for y: 0..r.height - 1 memcpy(frame.pixels.data + y * row, cast(*u8) mapped.pData + cast(s64) y * mapped.RowPitch, row);
ID3D11DeviceContext_Unmap(gpu.device_context, gpu.staging[slot], 0);
if r.options.cursor && staged.pointer_visible {
cursor_draw(frame.pixels.data, r.width, r.height, *r.dxgi.shape,
staged.pointer.x - cast(s32) gpu.box.left, staged.pointer.y - cast(s32) gpu.box.top);
}
frame.ticks = staged.ticks;
lock(*r.mutex);
r.queue[r.queue_count] = index;
r.queue_count += 1;
memcpy(r.last_frame.data, frame.pixels.data, frame.pixels.count);
r.stats.readback_ticks += ticks_now() - t;
unlock(*r.mutex);
signal(*r.ready);
}
record_encoder_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
while true {
wait_for(*r.ready, 100);
lock(*r.mutex);
index: s32 = -1;
if r.queue_count {
index = r.queue[0];
for 1..r.queue_count - 1 r.queue[it - 1] = r.queue[it];
r.queue_count -= 1;
}
unlock(*r.mutex);
if index < 0 {
if atomic_read(*r.capture_ended) break;
continue;
}
frame := *r.frames[index];
pts := max(frame.ticks, 0) * 90000 / r.frequency; // frame.ticks is on the timeline already
t := ticks_now();
ok, error := encoder_write(*r.encoder, frame.pixels.data, pts);
lock(*r.mutex);
r.free[r.free_count] = index;
r.free_count += 1;
r.stats.encode_ticks += ticks_now() - t;
if ok r.stats.written += 1;
r.stats.bytes = r.encoder.bytes;
unlock(*r.mutex);
if !ok {
record_fail(r, error);
// Frames still come until the capture thread sees it; they are let go.
while !atomic_read(*r.capture_ended) {
wait_for(*r.ready, 100);
lock(*r.mutex);
for 0..r.queue_count - 1 { r.free[r.free_count] = r.queue[it]; r.free_count += 1; }
r.queue_count = 0;
unlock(*r.mutex);
}
break;
}
}
return 0;
}
// The audio thread: opens the sources asked for, waits for the encoder, then every 10 ms takes their
// packets into the mix and encodes what every source has covered.
record_audio_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
audio_thread_start();
defer audio_thread_end();
kinds: [2] Audio_Source_Kind = .[.SYSTEM, .MICROPHONE];
wanted: [2] bool = .[r.options.system_audio, r.options.microphone];
names, errors: [..] string; // every source heard, and every failure, for the stats
names.allocator, errors.allocator = context.default_allocator, context.default_allocator;
defer { for names free(it); array_free(names); for errors free(it); array_free(errors); }
note :: (list: *[..] string, text: string, field: *string) #expand {
for list.* if it == text return;
array_add(list, copy_string(text));
lock(*`r.mutex);
free(field.*);
field.* = join(..list.*, "; ");
unlock(*`r.mutex);
}
for 0..1 {
if !wanted[it] continue;
s := *r.audio_sources[it];
ok, error := audio_open(s, kinds[it]);
what := ifx kinds[it] == .SYSTEM then "system sound" else "microphone";
if ok { r.audio_open = true; note(*names, s.name, *r.stats.audio); }
else { audio_close(s); note(*errors, tprint("%: %", what, error), *r.stats.audio_errors); log_error("[record] No %: %", what, error); }
}
for r.audio_sources if !it.open atomic_swap(*r.audio_on[it_index], false);
if r.options.audio_track r.audio_open = true;
signal(*r.audio_opened);
wait_for(*r.audio_go);
if !r.audio_open || !r.encoder.audio || atomic_read(*r.stopping) {
for * r.audio_sources audio_free(it);
return 0;
}
mix: Audio_Mix;
mix.frame_size = r.encoder.audio_frame_size;
defer array_free(mix.samples);
while true {
stopping := atomic_read(*r.stopping);
// Each source, turned on or off from the toolbar: opened only while on.
for * source, index: r.audio_sources {
on := atomic_read(*r.audio_on[index]) && !stopping;
what := ifx kinds[index] == .SYSTEM then "System sound" else "Microphone";
if on && !source.open && !wanted[index] {
ok, error := audio_open(source, kinds[index]);
if ok {
log("[record] % on: %.", what, source.name);
note(*names, source.name, *r.stats.audio);
mix.started[index] = false;
} else {
audio_close(source);
log_error("[record] % could not be opened: %", what, error);
note(*errors, tprint("%: %", to_lower_copy(what,, temp), error), *r.stats.audio_errors);
atomic_swap(*r.audio_on[index], false);
atomic_swap(*r.audio_failed[index], true);
}
} else if !on && source.open {
log("[record] % off.", what);
audio_close(source);
source.lost_at = 0;
}
wanted[index] = source.open || (wanted[index] && on);
}
for * source, index: r.audio_sources {
if !wanted[index] continue;
if !source.open { // lost: opened again a second on, from the default device then
if source.lost_at && ticks_now() - source.lost_at >= r.frequency && !stopping {
if audio_open(source, kinds[index]) log("[record] % is back: %.", ifx kinds[index] == .SYSTEM then "System sound" else "The microphone", source.name);
else { audio_close(source); source.lost_at = ticks_now(); }
}
continue;
}
packets, lost := audio_take(source, r.frequency);
for packets if !atomic_read(*r.paused) audio_mix_add(*mix, r, index, it);
if lost {
log_error("[record] % was lost (unplugged, or the default device changed); trying again.", source.name);
audio_close(source);
source.lost_at = ticks_now();
}
}
ok, error := audio_mix_encode(*mix, r, final = stopping);
if !ok { record_fail(r, error); break; }
if stopping break;
reset_temporary_storage();
sleep_milliseconds(10);
}
for * r.audio_sources audio_free(it);
return 0;
}
// The sound so far, from its base: one AAC frame's worth is encoded at a time.
Audio_Mix :: struct {
samples: [..] float; // interleaved stereo, from base on
base: s64; // the frame (in samples, on the timeline) samples[0] is
next: [2] s64; // each source's next frame, so packets that follow on are not moved by jitter
started: [2] bool;
frame_size: s32;
}
AUDIO_SLACK :: AUDIO_RATE / 5; // 200 ms: how late a source's packets may come and still be in the mix
AUDIO_FOLLOW_ON :: AUDIO_RATE / 33; // 30 ms: a packet that close to where its source left off follows on
// A packet into the mix at its place on the timeline, added to what the other source put there.
audio_mix_add :: (mix: *Audio_Mix, r: *Recorder, source: s64, packet: Audio_Packet) {
at := record_timeline(r, packet.ticks) * AUDIO_RATE / r.frequency;
if mix.started[source] && abs(at - mix.next[source]) <= AUDIO_FOLLOW_ON at = mix.next[source];
mix.started[source] = true;
mix.next[source] = at + packet.frames;
end := (at + packet.frames - mix.base) * AUDIO_CHANNELS;
if end > mix.samples.count {
old := mix.samples.count;
array_resize(*mix.samples, end, initialize = false);
for old..end - 1 mix.samples[it] = 0;
}
if packet.silent || !packet.samples.count return;
for frame: max(at, mix.base)..at + packet.frames - 1 {
into := (frame - mix.base) * AUDIO_CHANNELS;
from := (frame - at) * AUDIO_CHANNELS;
mix.samples[into] += packet.samples[from];
mix.samples[into + 1] += packet.samples[from + 1];
}
}
// Encodes the frames every open source has covered: up to where each has sent, or, for one that has
// sent nothing lately (silence), to AUDIO_SLACK ago. final: everything sent.
audio_mix_encode :: (mix: *Audio_Mix, r: *Recorder, final: bool) -> ok: bool, error: string {
// Paused, the timeline stands still where the pause began.
now := ifx atomic_read(*r.paused) then atomic_read(*r.pause_began) else ticks_now();
now_frame := record_timeline(r, now) * AUDIO_RATE / r.frequency;
ready := now_frame - AUDIO_SLACK;
for source, index: r.audio_sources {
if !source.open continue;
covered := max(mix.next[index], now_frame - AUDIO_SLACK);
ready = min(ready, covered);
}
if final { ready = mix.base; for mix.next ready = max(ready, it); }
size := mix.frame_size;
frame := NewArray(size * AUDIO_CHANNELS, float, initialized = false,, temp);
done: s64;
while mix.base + size <= ready {
for i: 0..frame.count - 1 {
at := done * AUDIO_CHANNELS + i;
value := ifx at < mix.samples.count then mix.samples[at] else 0.0;
frame[i] = clamp(value, -1.0, 1.0);
}
ok, error := encoder_write_audio(*r.encoder, frame.data, mix.base);
if !ok return false, error;
mix.base += size;
done += size;
lock(*r.mutex); r.stats.audio_frames += 1; unlock(*r.mutex);
}
// What was encoded leaves the front.
gone := min(done * AUDIO_CHANNELS, mix.samples.count);
if gone {
memcpy(mix.samples.data, mix.samples.data + gone, (mix.samples.count - gone) * size_of(float));
mix.samples.count -= gone;
}
return true, "";
}
// Ends the recording from one of its threads; the first reason is kept.
record_fail :: (r: *Recorder, error: string) {
log_error("[record] %", error);
lock(*r.mutex);
if !r.error r.error = copy_string(error);
unlock(*r.mutex);
atomic_swap(*r.stopping, true);
}
rect_inside :: (inner: RECT, outer: RECT) -> bool {
return inner.left >= outer.left && inner.top >= outer.top && inner.right <= outer.right && inner.bottom <= outer.bottom;
}
#scope_export
#load "capture_gpu.jai";
#load "dxgi_source.jai";
#load "wgc_source.jai";
#load "encoder.jai"; #load "encoder.jai";
#load "remux.jai"; #load "remux.jai";
#load "audio_source.jai";
#load "cursor.jai";
#import "Basic"; #import "Basic";
#import "String"; #import "String";
#import "Atomics"; #import "Atomics";
#import "Math"; #import "Math";
#import "Thread"; #import "Thread";
#import "Windows";
#import "Windows_Utf8";
#import "dxgi";
#import "d3d11";
#import "ffmpeg"; #import "ffmpeg";

701
modules/Screen_Record/windows.jai Normal file
View File

@ -0,0 +1,701 @@
// The Windows recorder (module.jai has what is shared): moved here from module.jai unchanged.
//
// A region of one screen recorded to H.264 in an MP4, for the mcap app's screen
// recording and tools/record_probe.jai (video-capture-plan.html). Windows only.
//
// Two threads. The capture thread takes the screen with DXGI Desktop Duplication (dxgi_source.jai),
// or with Windows.Graphics.Capture (wgc_source.jai) where duplication cannot run, each on a D3D11
// device of its own; copies the region on the GPU; reads it back through a ring of staging textures,
// so Map never waits on the GPU (capture_gpu.jai); and, for duplication, draws the cursor in
// (cursor.jai). The encoder thread hands the frames to FFmpeg (encoder.jai). Between them is a pool of
// RECORD_BUFFERS frames; when the encoder falls RECORD_QUEUE frames behind, the oldest waiting is
// dropped and counted.
//
// A frame is taken at most fps times a second, and only when the screen or the cursor changed, with
// one a second while nothing does, so players can seek. Each carries the time it was taken, less the
// time spent paused (record_pause), so a pause leaves no gap in the video.
//
// Sound (system_audio, microphone) has a thread of its own (audio_source.jai's WASAPI): each packet is
// put on the same timeline as the frames, by when it was heard, less the same pauses, into one mix,
// and an AAC frame of it is encoded once every source has had its say up to there. A source that sends
// nothing (the computer playing nothing) is silence. Either source can be turned on and off while
// recording (record_set_audio); with audio_track, the MP4 has a sound track even when no source is on
// at the start, so one can be.
//
// recorder: Recorder;
// ok, error, fall_back := record_start(*recorder, .{path = "out.mp4", screen = 0});
// ...
// ok, error = record_stop(*recorder);
//
// With capture .AUTO (the default), Desktop Duplication is tried first and Windows.Graphics.Capture
// takes over when it cannot run: on a laptop with two GPUs, on a rotated screen, or with too many
// duplications of the screen running. fall_back is only returned for .DXGI, which does not fall back.
Record_Screen :: struct {
adapter_index, output_index: u32;
adapter: string; // "NVIDIA GeForce RTX 5080 Laptop GPU"
name: string; // "\\.\DISPLAY1"
desktop: RECT; // on the virtual desktop
rotation: DXGI_MODE_ROTATION;
monitor: HMONITOR;
}
Record_Capture :: enum u8 {
AUTO; // Desktop Duplication, else Windows.Graphics.Capture
DXGI; // Desktop Duplication only
WGC; // Windows.Graphics.Capture only
}
// For record-probe: paths a machine may not take by itself.
Record_Testing :: enum_flags u8 {
DXGI_UNSUPPORTED; // Desktop Duplication fails as on a laptop with two GPUs (DXGI_ERROR_UNSUPPORTED)
WGC_POLL; // Windows.Graphics.Capture polled, as when it will not take the FrameArrived handler
KEEP_BORDER; // Windows.Graphics.Capture's yellow border left on
LOSE_DUPLICATION; // Desktop Duplication lost every 3 s, as a mode change or the secure desktop loses it
}
Record_Options :: struct {
path: string; // the .mp4 written
screen: s32 = -1; // into record_list_screens(); -1: the one holding region
region: RECT; // on the virtual desktop; empty: the whole screen
fps: s32 = 30;
cursor := true;
encoder: string; // "" tries RECORD_ENCODERS in turn
capture: Record_Capture;
legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe
testing: Record_Testing;
system_audio: bool; // what the computer plays (the default output device)
microphone: bool; // the default input device; both at the start: record_set_audio turns them on and off
audio_track: bool; // a sound track whatever is on at the start, so a source can be turned on later
}
Record_Stats :: struct {
source: string; // "DXGI DuplicateOutput1 on <adapter>", "Windows.Graphics.Capture on <adapter>, border off"
fell_back_from: string; // why Desktop Duplication could not run, when it fell back
encoder: string;
width, height: s32;
captured: s64; // new images of the screen
pointer_moves: s64; // cursor changes with no new image (Desktop Duplication)
emitted: s64; // frames sent to be encoded
repeats: s64; // of them, the once-a-second repeat of an unchanged screen
written: s64; // encoded
dropped: s64; // the encoder was too far behind
lost: s64; // times the duplication was lost and made again
paused_seconds: float64; // left out of the video
audio: string; // the sound recorded: "Speakers (Realtek(R) Audio); Microphone (USB Audio)", or ""
audio_frames: s64; // AAC frames encoded
audio_errors: string; // sources that could not be opened, or were lost
copy_ticks, readback_ticks, encode_ticks: s64; // totals, in QueryPerformanceCounter ticks
bytes: s64; // encoded
seconds: float64; // from start to stop
}
Recorder :: struct {
options: Record_Options;
screen: Record_Screen;
local: RECT; // the region, in the screen's pixels
dxgi: Dxgi_Source; // the capture thread's: one of these is open
wgc: Wgc_Source;
using_wgc: bool;
encoder: Encoder;
width, height: s32;
frequency: s64; // QueryPerformanceCounter's
start_ticks: s64;
capture_thread, encoder_thread: Thread;
capture_started, encoder_started, audio_started: bool;
audio_thread: Thread;
audio_opened: Semaphore; // the audio thread has opened its sources
audio_go: Semaphore; // the encoder is open: the audio thread may write
audio_sources: [2] Audio_Source;
audio_open: bool; // any source opened, or audio_track: the MP4 has sound
audio_on: [2] bool; // each source is wanted (by Audio_Source_Kind): record_set_audio; cleared when it cannot be opened
audio_failed: [2] bool; // it could not be opened after being turned on: record_audio_failed says once
zero_ticks: s64; // the recording's time 0, for frames and sound alike
paused_ticks: s64; // time paused so far, for the capture and audio threads; set by record_pause
pause_began: s64;
running: bool;
stopping: bool; // set by record_stop; the capture thread ends, then the encoder
paused: bool; // set by record_pause; the capture thread takes nothing while it is
capture_ended: bool;
opened: Semaphore; // the capture thread has opened its source, or failed to
open_ok, open_fall_back: bool;
open_error: string;
mutex: Mutex;
ready: Semaphore; // signalled for each frame queued, and once when capture ends
frames: [RECORD_BUFFERS] Record_Frame;
free: [RECORD_BUFFERS] s32;
free_count: s32;
queue: [RECORD_BUFFERS] s32; // oldest first
queue_count: s32;
last_frame: [] u8; // the last frame queued, BGRA, for record_last_frame
stats: Record_Stats;
error: string; // why it stopped by itself; heap
}
Record_Frame :: struct {
pixels: [] u8; // BGRA, rows packed
ticks: s64;
}
// Opens the screen, the encoder and the file, and starts recording.
record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: string, fall_back: bool {
r.* = .{};
r.options = options;
r.options.path = copy_string(options.path);
r.options.encoder = copy_string(options.encoder);
if r.options.fps <= 0 r.options.fps = 30;
screens := record_list_screens();
if !screens.count return false, "no screen to record", false;
screen_index := options.screen;
region := options.region;
has_region := region.right > region.left && region.bottom > region.top;
if screen_index < 0 {
if !has_region screen_index = 0;
else for screens if rect_inside(region, it.desktop) { screen_index = xx it_index; break; }
if screen_index < 0 return false, "the region is not on one screen", false;
}
if screen_index >= screens.count return false, tprint("there is no screen %", screen_index), false;
r.screen = screens[screen_index];
r.screen.adapter = copy_string(r.screen.adapter);
r.screen.name = copy_string(r.screen.name);
if !has_region region = r.screen.desktop;
if !rect_inside(region, r.screen.desktop) return false, "the region is not on that screen", false;
// The minimum first, then even both ways, in the screen's pixels: H.264's 4:2:0 colour needs it.
// RECORD_MIN_WIDTH and _HEIGHT are even, so what passes stays at least that size.
#assert RECORD_MIN_WIDTH % 2 == 0 && RECORD_MIN_HEIGHT % 2 == 0;
if !record_size_allowed(region.right - region.left, region.bottom - region.top)
return false, tprint("% x % is smaller than % x %", region.right - region.left, region.bottom - region.top, RECORD_MIN_WIDTH, RECORD_MIN_HEIGHT), false;
d := r.screen.desktop;
r.local = .{region.left - d.left, region.top - d.top, region.right - d.left, region.bottom - d.top};
r.local.right = r.local.left + ((r.local.right - r.local.left) & cast(s32) ~1);
r.local.bottom = r.local.top + ((r.local.bottom - r.local.top) & cast(s32) ~1);
r.width = r.local.right - r.local.left;
r.height = r.local.bottom - r.local.top;
r.stats.width, r.stats.height = r.width, r.height;
init(*r.mutex, "record");
init(*r.ready);
init(*r.opened);
for * r.frames it.pixels = NewArray(cast(s64) r.width * r.height * 4, u8, initialized = false);
for 0..RECORD_BUFFERS - 1 r.free[it] = cast(s32) it;
r.free_count = RECORD_BUFFERS;
r.last_frame = NewArray(cast(s64) r.width * r.height * 4, u8);
frequency: LARGE_INTEGER;
QueryPerformanceFrequency(*frequency);
r.frequency = frequency.QuadPart;
r.start_ticks = ticks_now();
r.zero_ticks = r.start_ticks;
// The source is opened on the capture thread, which uses it: WinRT's objects belong to the
// apartment they are made in.
thread_init(*r.capture_thread, record_capture_proc, name = "record capture");
r.capture_started = true;
thread_start(*r.capture_thread, r);
wait_for(*r.opened);
if !r.open_ok {
error, fall_back := copy_string(r.open_error,, temp), r.open_fall_back;
record_free(r);
return false, error, fall_back;
}
// The sound's sources are opened on their own thread (COM, for WASAPI), before the encoder, which
// has a sound stream only when one opened.
r.audio_on = .[r.options.system_audio, r.options.microphone];
if r.options.system_audio || r.options.microphone || r.options.audio_track {
init(*r.audio_opened);
init(*r.audio_go);
thread_init(*r.audio_thread, record_audio_proc, name = "record audio");
r.audio_started = true;
thread_start(*r.audio_thread, r);
wait_for(*r.audio_opened);
}
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder, audio = r.audio_open);
if tried log("[record] Encoders that did not open: %", tried);
if !ok {
error = copy_string(error,, temp);
record_free(r);
return false, error, false;
}
r.stats.encoder = copy_string(r.encoder.name);
if r.audio_started signal(*r.audio_go);
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
r.encoder_started = true;
r.running = true;
thread_start(*r.encoder_thread, r);
return true, "", false;
}
// Stops, finishes the file and frees everything. ok is false when recording stopped by itself on an
// error, or the file could not be finished; what was recorded up to then is in the file all the same.
record_stop :: (r: *Recorder) -> ok: bool, error: string {
if !r.running return false, "not recording";
atomic_swap(*r.stopping, true);
thread_is_done(*r.capture_thread, -1);
thread_is_done(*r.encoder_thread, -1);
if r.audio_started thread_is_done(*r.audio_thread, -1);
r.running = false;
closed, close_error := encoder_close(*r.encoder);
r.stats.seconds = cast(float64) (ticks_now() - r.start_ticks) / r.frequency;
r.stats.bytes = r.encoder.bytes;
error := ifx r.error then copy_string(r.error,, temp) else "";
record_free(r);
if error return false, error;
if !closed return false, close_error;
return true, "";
}
// Pauses or resumes. While paused nothing is taken, and the time is left out of the video, sound and
// all: the time paused is kept here, once, for the capture and audio threads both.
record_pause :: (r: *Recorder, pause: bool) {
if !r.running || atomic_read(*r.paused) == pause return;
now := ticks_now();
if pause {
atomic_write(*r.pause_began, now);
atomic_swap(*r.paused, true);
} else {
atomic_add(*r.paused_ticks, now - atomic_read(*r.pause_began)); // before paused clears, so no thread sees it unpaused with the old total
atomic_swap(*r.paused, false);
}
}
// Turns a source of sound on or off while recording. It is opened only while on, so Windows shows the
// microphone in use only then. False when the recording has no sound track to put it in.
record_set_audio :: (r: *Recorder, kind: Audio_Source_Kind, on: bool) -> bool {
if !r.running || !r.audio_open return false;
atomic_swap(*r.audio_on[kind], on);
return true;
}
record_audio_on :: (r: *Recorder, kind: Audio_Source_Kind) -> bool {
return r.running && atomic_read(*r.audio_on[kind]);
}
// Whether a source could not be opened when turned on (it is off again then), and why; once.
record_audio_failed :: (r: *Recorder, kind: Audio_Source_Kind) -> failed: bool, error: string {
if !r.running || !atomic_read(*r.audio_failed[kind]) return false, "";
atomic_swap(*r.audio_failed[kind], false);
lock(*r.mutex);
defer unlock(*r.mutex);
return true, copy_string(r.stats.audio_errors,, temp);
}
// Ticks on the recording's timeline: time 0 at its start, less the time paused.
record_timeline :: (r: *Recorder, ticks: s64) -> s64 {
return ticks - r.zero_ticks - atomic_read(*r.paused_ticks);
}
record_paused :: (r: *Recorder) -> bool {
return r.running && atomic_read(*r.paused);
}
// As they are now; strings are the recorder's.
record_stats :: (r: *Recorder) -> Record_Stats {
if !r.running return r.stats;
lock(*r.mutex);
defer unlock(*r.mutex);
return r.stats;
}
// The last frame queued, BGRA (temp), cursor and all.
record_last_frame :: (r: *Recorder) -> pixels: [] u8, width: s32, height: s32 {
if r.running lock(*r.mutex);
pixels := array_copy(r.last_frame,, temp);
if r.running unlock(*r.mutex);
return pixels, r.width, r.height;
}
// Whether recording stopped by itself (the screen went away, the encoder failed); why, then.
record_failed :: (r: *Recorder) -> bool, string {
lock(*r.mutex);
defer unlock(*r.mutex);
return r.error.count > 0, r.error;
}
#scope_file
Staged :: struct { pending: bool; ticks: s64; pointer: POINT; pointer_visible: bool; }
// Stops the capture thread if it runs, and frees what record_start made. The stats' strings are kept.
record_free :: (r: *Recorder) {
atomic_swap(*r.stopping, true);
if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; }
if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; }
if r.audio_started {
signal(*r.audio_go); // in case it waits for an encoder that did not open
thread_is_done(*r.audio_thread, -1);
thread_deinit(*r.audio_thread);
destroy(*r.audio_opened);
destroy(*r.audio_go);
r.audio_started = false;
}
if r.encoder.format encoder_close(*r.encoder);
for * r.frames array_free(it.pixels);
array_free(r.last_frame);
r.last_frame = .[];
destroy(*r.opened);
destroy(*r.ready);
destroy(*r.mutex);
free(r.open_error);
free(r.error);
r.open_error, r.error = "", "";
}
// Desktop Duplication, unless asked for Windows.Graphics.Capture, which takes over when duplication
// cannot run here and the options allow.
record_open_source :: (r: *Recorder) -> bool {
if r.options.capture != .WGC {
ok, error, fall_back := dxgi_open(*r.dxgi, r.screen, r.local, r.options.legacy_duplicate);
if ok && r.options.testing & .DXGI_UNSUPPORTED {
ok, error, fall_back = false, "Desktop Duplication (testing): 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", true;
}
if ok {
r.stats.source = sprint("DXGI % on %", r.dxgi.method, r.screen.adapter);
return true;
}
dxgi_close(*r.dxgi);
if r.options.capture == .DXGI || !fall_back {
r.open_error, r.open_fall_back = copy_string(error), fall_back;
return false;
}
log("[record] Desktop Duplication cannot run here (%); Windows.Graphics.Capture instead.", error);
r.stats.fell_back_from = copy_string(error);
}
ok, error := wgc_open(*r.wgc, r.screen, r.local, r.options.cursor, r.options.testing);
if !ok {
wgc_close(*r.wgc);
r.open_error = copy_string(error);
return false;
}
r.using_wgc = true;
r.stats.source = sprint("Windows.Graphics.Capture on %, %%", r.screen.adapter, r.wgc.border, ifx r.wgc.polling then ", polled" else "");
return true;
}
record_capture_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
// Aware of each monitor's DPI, whatever the process is: Desktop Duplication refuses a thread that is
// not (DXGI_ERROR_UNSUPPORTED), and the mcap app's ImGui makes only its main thread aware.
gpu_thread_dpi_aware();
wgc_thread_start();
defer wgc_thread_end();
r.open_ok = record_open_source(r);
signal(*r.opened);
if !r.open_ok return 0;
gpu: *Capture_Gpu = ifx r.using_wgc then *r.wgc.gpu else *r.dxgi.gpu;
defer if r.using_wgc wgc_close(*r.wgc); else dxgi_close(*r.dxgi);
// A change is sent as soon as it comes once its turn is due: turns are a frame's time apart on a
// schedule that moves on a frame at a time, so the rate never passes fps, and a tenth of a frame
// early counts as on time, so a 60 Hz screen's frames, which jitter, all make a 60 fps cap.
period := r.frequency / r.options.fps;
early := period / 10;
turn := ticks_now();
last_emit: s64 = -r.frequency;
have_image, changed := false, false;
staged: [RECORD_STAGING] Staged;
emits: s64;
was_paused := false;
last_lost := ticks_now(); // for Record_Testing.LOSE_DUPLICATION
while !atomic_read(*r.stopping) {
now := ticks_now();
due := ifx changed then turn - early else last_emit + r.frequency; // the change, or the repeat
wait := cast(u32) clamp((due - now) * 1000 / r.frequency, 0, 100);
next: Capture_Next;
error: string;
if r.using_wgc next, error = wgc_next(*r.wgc, wait);
else next, error = dxgi_next(*r.dxgi, wait);
if r.options.testing & .LOSE_DUPLICATION && !r.using_wgc && ticks_now() - last_lost >= 3 * r.frequency {
next, last_lost = Capture_Next.LOST, ticks_now();
}
if next == {
case .FAILED;
record_fail(r, error);
break;
case .LOST;
// A mode change, the secure desktop, a game going fullscreen: duplicated again, and on.
lock(*r.mutex); r.stats.lost += 1; unlock(*r.mutex);
while !atomic_read(*r.stopping) {
hr := dxgi_duplicate(*r.dxgi);
if SUCCEEDED(hr) break;
if !hresult_is(hr, DXGI_ERROR_ACCESS_LOST) && !hresult_is(hr, 0x80070005) && !hresult_is(hr, DXGI_ERROR_NOT_CURRENTLY_AVAILABLE) {
record_fail(r, hresult_text("the screen could not be duplicated again", hr));
break;
}
sleep_milliseconds(250);
}
if atomic_read(*r.stopping) || r.error break;
continue;
case .IMAGE; #through;
case .IMAGE_AND_POINTER;
have_image, changed = true, true;
lock(*r.mutex);
r.stats.captured += 1;
r.stats.copy_ticks = gpu.copy_ticks;
unlock(*r.mutex);
case .POINTER;
if r.options.cursor && have_image {
changed = true;
lock(*r.mutex); r.stats.pointer_moves += 1; unlock(*r.mutex);
}
}
// Paused: the source is still drained, so it never stalls, but nothing is sent. On resuming, the
// screen as it is now is sent at once.
now = ticks_now();
if atomic_read(*r.paused) {
was_paused = true;
lock(*r.mutex); r.stats.paused_seconds = cast(float64) (atomic_read(*r.paused_ticks) + now - atomic_read(*r.pause_began)) / r.frequency; unlock(*r.mutex);
continue;
}
if was_paused {
was_paused = false;
changed = true;
turn = now;
lock(*r.mutex); r.stats.paused_seconds = cast(float64) atomic_read(*r.paused_ticks) / r.frequency; unlock(*r.mutex);
}
if !have_image continue;
repeat := !changed && now - last_emit >= r.frequency;
if !(changed && now >= turn - early) && !repeat continue;
turn = max(turn + period, now - period); // after a quiet spell, no burst to catch up
// Copied to staging now; read back two frames on, when the GPU is done with it.
slot := emits % RECORD_STAGING;
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
ID3D11DeviceContext_CopyResource(gpu.device_context, gpu.staging[slot], gpu.latest);
staged[slot] = .{pending = true, ticks = record_timeline(r, now)};
if !r.using_wgc staged[slot].pointer, staged[slot].pointer_visible = r.dxgi.pointer_position, r.dxgi.pointer_visible;
emits += 1;
last_emit = now;
changed = false;
lock(*r.mutex);
r.stats.emitted += 1;
if repeat r.stats.repeats += 1;
unlock(*r.mutex);
}
// What is still staged, oldest first.
for 0..RECORD_STAGING - 1 {
slot := (emits + it) % RECORD_STAGING;
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
}
atomic_swap(*r.capture_ended, true);
signal(*r.ready);
return 0;
}
// Copies a staging texture into a free frame, draws the cursor (Desktop Duplication leaves it out)
// and queues it.
record_read_back :: (r: *Recorder, gpu: *Capture_Gpu, slot: s64, staged: Staged) {
t := ticks_now();
lock(*r.mutex);
if !r.free_count { // every frame is waiting: drop the oldest
r.free[r.free_count] = r.queue[0];
r.free_count += 1;
for 1..r.queue_count - 1 r.queue[it - 1] = r.queue[it];
r.queue_count -= 1;
r.stats.dropped += 1;
}
r.free_count -= 1;
index := r.free[r.free_count];
unlock(*r.mutex);
frame := *r.frames[index];
mapped: D3D11_MAPPED_SUBRESOURCE;
hr := ID3D11DeviceContext_Map(gpu.device_context, gpu.staging[slot], 0, .READ, 0, *mapped);
if FAILED(hr) {
lock(*r.mutex); r.free[r.free_count] = index; r.free_count += 1; unlock(*r.mutex);
record_fail(r, hresult_text("reading the frame back", hr));
return;
}
row := cast(s64) r.width * 4;
for y: 0..r.height - 1 memcpy(frame.pixels.data + y * row, cast(*u8) mapped.pData + cast(s64) y * mapped.RowPitch, row);
ID3D11DeviceContext_Unmap(gpu.device_context, gpu.staging[slot], 0);
if r.options.cursor && staged.pointer_visible {
cursor_draw(frame.pixels.data, r.width, r.height, *r.dxgi.shape,
staged.pointer.x - cast(s32) gpu.box.left, staged.pointer.y - cast(s32) gpu.box.top);
}
frame.ticks = staged.ticks;
lock(*r.mutex);
r.queue[r.queue_count] = index;
r.queue_count += 1;
memcpy(r.last_frame.data, frame.pixels.data, frame.pixels.count);
r.stats.readback_ticks += ticks_now() - t;
unlock(*r.mutex);
signal(*r.ready);
}
record_encoder_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
while true {
wait_for(*r.ready, 100);
lock(*r.mutex);
index: s32 = -1;
if r.queue_count {
index = r.queue[0];
for 1..r.queue_count - 1 r.queue[it - 1] = r.queue[it];
r.queue_count -= 1;
}
unlock(*r.mutex);
if index < 0 {
if atomic_read(*r.capture_ended) break;
continue;
}
frame := *r.frames[index];
pts := max(frame.ticks, 0) * 90000 / r.frequency; // frame.ticks is on the timeline already
t := ticks_now();
ok, error := encoder_write(*r.encoder, frame.pixels.data, pts);
lock(*r.mutex);
r.free[r.free_count] = index;
r.free_count += 1;
r.stats.encode_ticks += ticks_now() - t;
if ok r.stats.written += 1;
r.stats.bytes = r.encoder.bytes;
unlock(*r.mutex);
if !ok {
record_fail(r, error);
// Frames still come until the capture thread sees it; they are let go.
while !atomic_read(*r.capture_ended) {
wait_for(*r.ready, 100);
lock(*r.mutex);
for 0..r.queue_count - 1 { r.free[r.free_count] = r.queue[it]; r.free_count += 1; }
r.queue_count = 0;
unlock(*r.mutex);
}
break;
}
}
return 0;
}
// The audio thread: opens the sources asked for, waits for the encoder, then every 10 ms takes their
// packets into the mix and encodes what every source has covered.
record_audio_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data;
audio_thread_start();
defer audio_thread_end();
kinds: [2] Audio_Source_Kind = .[.SYSTEM, .MICROPHONE];
wanted: [2] bool = .[r.options.system_audio, r.options.microphone];
names, errors: [..] string; // every source heard, and every failure, for the stats
names.allocator, errors.allocator = context.default_allocator, context.default_allocator;
defer { for names free(it); array_free(names); for errors free(it); array_free(errors); }
note :: (list: *[..] string, text: string, field: *string) #expand {
for list.* if it == text return;
array_add(list, copy_string(text));
lock(*`r.mutex);
free(field.*);
field.* = join(..list.*, "; ");
unlock(*`r.mutex);
}
for 0..1 {
if !wanted[it] continue;
s := *r.audio_sources[it];
ok, error := audio_open(s, kinds[it]);
what := ifx kinds[it] == .SYSTEM then "system sound" else "microphone";
if ok { r.audio_open = true; note(*names, s.name, *r.stats.audio); }
else { audio_close(s); note(*errors, tprint("%: %", what, error), *r.stats.audio_errors); log_error("[record] No %: %", what, error); }
}
for r.audio_sources if !it.open atomic_swap(*r.audio_on[it_index], false);
if r.options.audio_track r.audio_open = true;
signal(*r.audio_opened);
wait_for(*r.audio_go);
if !r.audio_open || !r.encoder.audio || atomic_read(*r.stopping) {
for * r.audio_sources audio_free(it);
return 0;
}
mix: Audio_Mix;
mix.frame_size = r.encoder.audio_frame_size;
defer array_free(mix.samples);
while true {
stopping := atomic_read(*r.stopping);
// Each source, turned on or off from the toolbar: opened only while on.
for * source, index: r.audio_sources {
on := atomic_read(*r.audio_on[index]) && !stopping;
what := ifx kinds[index] == .SYSTEM then "System sound" else "Microphone";
if on && !source.open && !wanted[index] {
ok, error := audio_open(source, kinds[index]);
if ok {
log("[record] % on: %.", what, source.name);
note(*names, source.name, *r.stats.audio);
mix.started[index] = false;
} else {
audio_close(source);
log_error("[record] % could not be opened: %", what, error);
note(*errors, tprint("%: %", to_lower_copy(what,, temp), error), *r.stats.audio_errors);
atomic_swap(*r.audio_on[index], false);
atomic_swap(*r.audio_failed[index], true);
}
} else if !on && source.open {
log("[record] % off.", what);
audio_close(source);
source.lost_at = 0;
}
wanted[index] = source.open || (wanted[index] && on);
}
for * source, index: r.audio_sources {
if !wanted[index] continue;
if !source.open { // lost: opened again a second on, from the default device then
if source.lost_at && ticks_now() - source.lost_at >= r.frequency && !stopping {
if audio_open(source, kinds[index]) log("[record] % is back: %.", ifx kinds[index] == .SYSTEM then "System sound" else "The microphone", source.name);
else { audio_close(source); source.lost_at = ticks_now(); }
}
continue;
}
packets, lost := audio_take(source, r.frequency);
for packets if !atomic_read(*r.paused) audio_mix_add(*mix, r, index, it);
if lost {
log_error("[record] % was lost (unplugged, or the default device changed); trying again.", source.name);
audio_close(source);
source.lost_at = ticks_now();
}
}
ok, error := audio_mix_encode(*mix, r, final = stopping);
if !ok { record_fail(r, error); break; }
if stopping break;
reset_temporary_storage();
sleep_milliseconds(10);
}
for * r.audio_sources audio_free(it);
return 0;
}
// Ends the recording from one of its threads; the first reason is kept.
record_fail :: (r: *Recorder, error: string) {
log_error("[record] %", error);
lock(*r.mutex);
if !r.error r.error = copy_string(error);
unlock(*r.mutex);
atomic_swap(*r.stopping, true);
}
rect_inside :: (inner: RECT, outer: RECT) -> bool {
return inner.left >= outer.left && inner.top >= outer.top && inner.right <= outer.right && inner.bottom <= outer.bottom;
}
#scope_export
#load "capture_gpu.jai";
#load "dxgi_source.jai";
#load "wgc_source.jai";
#load "audio_source.jai";
#load "cursor.jai";
#import "Windows";
#import "Windows_Utf8";
#import "dxgi";
#import "d3d11";

3
src/App.jai
View File

@ -33,6 +33,7 @@ USING_WINDOWS :: OS == .WINDOWS;
#if OS == .WINDOWS #load "app/screen_capture_win32.jai"; #if OS == .WINDOWS #load "app/screen_capture_win32.jai";
#if OS == .MACOS #load "app/screen_capture_macos.jai"; #if OS == .MACOS #load "app/screen_capture_macos.jai";
#if OS == .WINDOWS #load "app/screen_record_win32.jai"; #if OS == .WINDOWS #load "app/screen_record_win32.jai";
#if OS == .MACOS #load "app/screen_record_macos.jai";
#load "base/circle_buffer.jai"; #load "base/circle_buffer.jai";
#load "base/debug.jai"; #load "base/debug.jai";
@ -79,7 +80,7 @@ Shared_Channel :: #import "Shared_Memory_Channel";
#import "Capture_Text"; #import "Capture_Text";
#import "Base64"; #import "Base64";
#import "onnxruntime"; #import "onnxruntime";
#if OS == .WINDOWS #import "Screen_Record"; // screen recording: Desktop Duplication or Windows.Graphics.Capture, to NVENC #if OS == .WINDOWS || OS == .MACOS #import "Screen_Record"; // screen recording: Desktop Duplication or Windows.Graphics.Capture, to NVENC
// Basic's arenas import Secure_Memory at its defaults; this import adds its test suite. // Basic's arenas import Secure_Memory at its defaults; this import adds its test suite.
using Secure_Memory :: #import "Secure_Memory"()(LOAD_TESTS=COMPILE_TESTS); using Secure_Memory :: #import "Secure_Memory"()(LOAD_TESTS=COMPILE_TESTS);

97
src/app/screen_capture_macos.jai
View File

@ -7,6 +7,11 @@
// selection's size, so a region can be chosen to the pixel; the arrow keys move the cursor a pixel at // selection's size, so a region can be chosen to the pixel; the arrow keys move the cursor a pixel at
// a time (ten with Shift). Escape or a right click cancels; Return takes what is dragged. // a time (ten with Shift). Escape or a right click cancels; Return takes what is dragged.
// //
// For a recording (video), the outline and magnifier are red, and a region smaller than
// RECORD_MIN_WIDTH x RECORD_MIN_HEIGHT is outlined dashed, with the minimum in the label, and letting
// go of it takes nothing: the overlay stays to drag again. No pixels are copied out for a recording,
// which app/screen_record.jai has ScreenCaptureKit stream from the display and region the shot names.
//
// A drag stays on the screen it began on: its pixels are that screen's, two to a point on a Retina // A drag stays on the screen it began on: its pixels are that screen's, two to a point on a Retina
// screen. Everything is worked out in each screen's own pixels, which the cursor's position in points // screen. Everything is worked out in each screen's own pixels, which the cursor's position in points
// is turned into. // is turned into.
@ -30,6 +35,13 @@ Screen_Shot :: struct {
window_title: string; // the window under the region's middle; heap, as are the two below window_title: string; // the window under the region's middle; heap, as are the two below
app_name: string; // its app ("Safari") app_name: string; // its app ("Safari")
app_path: string; // its app's bundle, or executable app_path: string; // its app's bundle, or executable
// The display it is on, and the region in that display's points from its top left: what a
// recording streams (modules/Screen_Record).
display_id: u32;
scale: float64; // its pixels a point
local_x, local_y, local_width, local_height: float64;
screen_left, screen_top, screen_width, screen_height: float64; // the display, in points from the main screen's top left
} }
screen_shot_free :: (shot: *Screen_Shot) { screen_shot_free :: (shot: *Screen_Shot) {
@ -41,12 +53,11 @@ screen_shot_free :: (shot: *Screen_Shot) {
shot.* = .{}; shot.* = .{};
} }
// Freezes the screens and lets a region be chosen. With immediate, no overlay: the whole screen under // Freezes the screens and lets a region be chosen: for a still, or with video for a recording. With
// the cursor is taken at once. ok is false when it was cancelled (error "") or could not start (error // immediate, no overlay: the whole screen under the cursor is taken at once. ok is false when it was
// says why). Recording (video) is not on macOS yet. // cancelled (error "") or could not start (error says why).
platform_screen_capture_select :: (video := false, immediate := false) -> Screen_Shot, ok: bool, error: string { platform_screen_capture_select :: (video := false, immediate := false) -> Screen_Shot, ok: bool, error: string {
shot: Screen_Shot; shot: Screen_Shot;
if video return shot, false, "screen recording is not on macOS yet";
init_objective_c(); init_objective_c();
pool := objc_new(NSAutoreleasePool); pool := objc_new(NSAutoreleasePool);
defer NSAutoreleasePool.drain(pool); defer NSAutoreleasePool.drain(pool);
@ -64,6 +75,7 @@ platform_screen_capture_select :: (video := false, immediate := false) -> Screen
ok, error := capture_freeze(); ok, error := capture_freeze();
defer capture_release(); defer capture_release();
if !ok return shot, false, error; if !ok return shot, false, error;
capture.video = video;
if immediate { if immediate {
x, y := capture_cursor_points(); x, y := capture_cursor_points();
@ -85,11 +97,18 @@ platform_screen_capture_select :: (video := false, immediate := false) -> Screen
shot.left = cast(s32) (screen.left + r.left / screen.scale); shot.left = cast(s32) (screen.left + r.left / screen.scale);
shot.top = cast(s32) (screen.top + r.top / screen.scale); shot.top = cast(s32) (screen.top + r.top / screen.scale);
shot.whole_screen = capture.whole_screen; shot.whole_screen = capture.whole_screen;
shot.pixels = NewArray(cast(s64) shot.width * shot.height * 4, u8, initialized = false,, context.default_allocator); shot.display_id = screen.display_id;
row_bytes := cast(s64) shot.width * 4; shot.scale = screen.scale;
for y: 0..shot.height - 1 { shot.screen_left, shot.screen_top, shot.screen_width, shot.screen_height = screen.left, screen.top, screen.width, screen.height;
from := screen.pixels + (cast(s64) (r.top + y) * screen.pixel_width + r.left) * 4; shot.local_x, shot.local_y = r.left / screen.scale, r.top / screen.scale;
memcpy(shot.pixels.data + y * row_bytes, from, row_bytes); shot.local_width, shot.local_height = (r.right - r.left) / screen.scale, (r.bottom - r.top) / screen.scale;
if !video {
shot.pixels = NewArray(cast(s64) shot.width * shot.height * 4, u8, initialized = false,, context.default_allocator);
row_bytes := cast(s64) shot.width * 4;
for y: 0..shot.height - 1 {
from := screen.pixels + (cast(s64) (r.top + y) * screen.pixel_width + r.left) * 4;
memcpy(shot.pixels.data + y * row_bytes, from, row_bytes);
}
} }
// The window under the region's middle, now that the overlay is gone; else the app that was in front. // The window under the region's middle, now that the overlay is gone; else the app that was in front.
@ -121,6 +140,7 @@ Pixel_Rect :: struct { left, top, right, bottom: s32; }
capture: struct { capture: struct {
screens: [..] Capture_Screen; screens: [..] Capture_Screen;
video: bool; // choosing a region to record
cursor_screen: s64; // the screen the cursor is on, and its pixel there cursor_screen: s64; // the screen the cursor is on, and its pixel there
cursor: Pixel; cursor: Pixel;
dragging: bool; dragging: bool;
@ -141,6 +161,9 @@ CLICK_SLOP :: 3; // a drag smaller than this either way, in pixels, is
DIM :: 0.5; // how dark the frozen screens are DIM :: 0.5; // how dark the frozen screens are
ACCENT :: Vector4.{1, 0.647, 0, 1}; // orange, as on Windows ACCENT :: Vector4.{1, 0.647, 0, 1}; // orange, as on Windows
RECORD :: Vector4.{0.91, 0.157, 0.157, 1}; // red: choosing a region to record
DASH : float64 : 6; // a too-small region's dashes and gaps, in points
GAP : float64 : 4;
LABEL :: Vector4.{0.125, 0.125, 0.125, 0.92}; LABEL :: Vector4.{0.125, 0.125, 0.125, 0.92};
TEXT :: Vector4.{1, 1, 1, 1}; TEXT :: Vector4.{1, 1, 1, 1};
@ -358,6 +381,11 @@ capture_take :: () {
if capture.dragging { if capture.dragging {
selection := capture_selection(); selection := capture_selection();
if selection.right - selection.left >= CLICK_SLOP || selection.bottom - selection.top >= CLICK_SLOP { if selection.right - selection.left >= CLICK_SLOP || selection.bottom - selection.top >= CLICK_SLOP {
if capture.video && !record_size_allowed(selection.right - selection.left, selection.bottom - selection.top) {
capture.dragging = false; // too small to record: nothing taken, the overlay stays
capture_update();
return;
}
capture.result_screen = capture.anchor_screen; capture.result_screen = capture.anchor_screen;
capture.result = selection; capture.result = selection;
capture.dragging = false; capture.dragging = false;
@ -432,14 +460,24 @@ magnifier_layout :: (screen: *Capture_Screen) -> magnifier: CGRect, label: CGRec
label_text :: (screen: *Capture_Screen) -> string { label_text :: (screen: *Capture_Screen) -> string {
if capture.dragging { if capture.dragging {
s := capture_selection(); s := capture_selection();
return tprint("% × %", s.right - s.left, s.bottom - s.top); text := tprint("% × %", s.right - s.left, s.bottom - s.top);
if capture_too_small(s) text = tprint("% · at least % × %", text, RECORD_MIN_WIDTH, RECORD_MIN_HEIGHT);
return text;
} }
return tprint("%, %", capture.cursor.x, capture.cursor.y); return tprint("%, %", capture.cursor.x, capture.cursor.y);
} }
// To record, smaller than the minimum (and not a click, which takes the whole screen).
capture_too_small :: (s: Pixel_Rect) -> bool {
w, h := s.right - s.left, s.bottom - s.top;
if !capture.video || (w < CLICK_SLOP && h < CLICK_SLOP) return false;
return !record_size_allowed(w, h);
}
// drawRect: for screen index's view, in its points (flipped: y goes down). // drawRect: for screen index's view, in its points (flipped: y goes down).
capture_draw :: (index: s64, cg: CGContextRef) { capture_draw :: (index: s64, cg: CGContextRef) {
screen := *capture.screens[index]; screen := *capture.screens[index];
accent := ifx capture.video then RECORD else ACCENT;
whole := CGRect.{.{0, 0}, .{screen.width, screen.height}}; whole := CGRect.{.{0, 0}, .{screen.width, screen.height}};
draw_image(cg, screen.image, whole); draw_image(cg, screen.image, whole);
CGContextSetRGBFillColor(cg, 0, 0, 0, DIM); CGContextSetRGBFillColor(cg, 0, 0, 0, DIM);
@ -455,18 +493,35 @@ capture_draw :: (index: s64, cg: CGContextRef) {
CGContextClipToRect(cg, s); CGContextClipToRect(cg, s);
draw_image(cg, screen.image, whole); draw_image(cg, screen.image, whole);
CGContextRestoreGState(cg); CGContextRestoreGState(cg);
// The outline just outside it, so every pixel taken shows as it is. // The outline just outside it, so every pixel taken shows as it is; dashed when too small to record.
fill(cg, .{.{s.origin.x - BORDER, s.origin.y - BORDER}, .{s.size.width + 2 * BORDER, BORDER}}, ACCENT); dashed := capture_too_small(capture_selection());
fill(cg, .{.{s.origin.x - BORDER, s.origin.y + s.size.height}, .{s.size.width + 2 * BORDER, BORDER}}, ACCENT); outline := CGRect.[
fill(cg, .{.{s.origin.x - BORDER, s.origin.y}, .{BORDER, s.size.height}}, ACCENT); .{.{s.origin.x - BORDER, s.origin.y - BORDER}, .{s.size.width + 2 * BORDER, BORDER}},
fill(cg, .{.{s.origin.x + s.size.width, s.origin.y}, .{BORDER, s.size.height}}, ACCENT); .{.{s.origin.x - BORDER, s.origin.y + s.size.height}, .{s.size.width + 2 * BORDER, BORDER}},
.{.{s.origin.x - BORDER, s.origin.y}, .{BORDER, s.size.height}},
.{.{s.origin.x + s.size.width, s.origin.y}, .{BORDER, s.size.height}},
];
for outline {
if !dashed { fill(cg, it, accent); continue; }
across := it.size.width > it.size.height;
start := ifx across then it.origin.x else it.origin.y;
end := start + (ifx across then it.size.width else it.size.height);
position := start;
while position < end {
piece := it;
if across { piece.origin.x = position; piece.size.width = min(DASH, end - position); }
else { piece.origin.y = position; piece.size.height = min(DASH, end - position); }
fill(cg, piece, accent);
position += DASH + GAP;
}
}
} }
if index != capture.cursor_screen return; if index != capture.cursor_screen return;
pixel := 1 / screen.scale; pixel := 1 / screen.scale;
cx, cy := capture.cursor.x * pixel, capture.cursor.y * pixel; cx, cy := capture.cursor.x * pixel, capture.cursor.y * pixel;
if !capture.dragging { if !capture.dragging {
fill(cg, .{.{0, cy}, .{screen.width, pixel}}, ACCENT); fill(cg, .{.{0, cy}, .{screen.width, pixel}}, accent);
fill(cg, .{.{cx, 0}, .{pixel, screen.height}}, ACCENT); fill(cg, .{.{cx, 0}, .{pixel, screen.height}}, accent);
} }
// The magnifier: the pixels around the cursor, as copied, each MAGNIFIER_ZOOM points wide; past // The magnifier: the pixels around the cursor, as copied, each MAGNIFIER_ZOOM points wide; past
@ -491,10 +546,10 @@ capture_draw :: (index: s64, cg: CGContextRef) {
} }
// The pixel under the cursor, outlined. // The pixel under the cursor, outlined.
c := CGRect.{.{inner.origin.x + half * MAGNIFIER_ZOOM, inner.origin.y + half * MAGNIFIER_ZOOM}, .{MAGNIFIER_ZOOM, MAGNIFIER_ZOOM}}; c := CGRect.{.{inner.origin.x + half * MAGNIFIER_ZOOM, inner.origin.y + half * MAGNIFIER_ZOOM}, .{MAGNIFIER_ZOOM, MAGNIFIER_ZOOM}};
fill(cg, .{.{c.origin.x - 1, c.origin.y - 1}, .{c.size.width + 2, 1}}, ACCENT); fill(cg, .{.{c.origin.x - 1, c.origin.y - 1}, .{c.size.width + 2, 1}}, accent);
fill(cg, .{.{c.origin.x - 1, c.origin.y + c.size.height}, .{c.size.width + 2, 1}}, ACCENT); fill(cg, .{.{c.origin.x - 1, c.origin.y + c.size.height}, .{c.size.width + 2, 1}}, accent);
fill(cg, .{.{c.origin.x - 1, c.origin.y}, .{1, c.size.height}}, ACCENT); fill(cg, .{.{c.origin.x - 1, c.origin.y}, .{1, c.size.height}}, accent);
fill(cg, .{.{c.origin.x + c.size.width, c.origin.y}, .{1, c.size.height}}, ACCENT); fill(cg, .{.{c.origin.x + c.size.width, c.origin.y}, .{1, c.size.height}}, accent);
// Under it, the position, or the selection's size. // Under it, the position, or the selection's size.
fill(cg, label, LABEL); fill(cg, label, LABEL);

55
src/app/screen_record.jai
View File

@ -1,6 +1,7 @@
// Screen recording (video-capture-plan.html): Ctrl+PrintScreen (the record_region global hotkey, from // Screen recording (video-capture-plan.html, macos-plan phase 7): Ctrl+PrintScreen on Windows,
// any app, even with mcap hidden in the tray) freezes the screens as PrintScreen does, in the // Opt+Cmd+Shift+2 on macOS (the record_region global hotkey, from any app, even with mcap hidden in the
// overlay's video mode (app/screen_capture_win32.jai): a region of one screen, at least 240 x 180, or // tray) freezes the screens as a still's key does, in the overlay's video mode
// (app/screen_capture_win32.jai, app/screen_capture_macos.jai): a region of one screen, at least 240 x 180, or
// a click for the whole screen. It is recorded with modules/Screen_Record to an MP4 in the temporary // a click for the whole screen. It is recorded with modules/Screen_Record to an MP4 in the temporary
// folder until Ctrl+PrintScreen is pressed again. The MP4 then goes into the library as a video .mcap // folder until Ctrl+PrintScreen is pressed again. The MP4 then goes into the library as a video .mcap
// in SCREEN_CAPTURE_FOLDER, beside the stills, through app/screen_capture.jai's jobs: the MP4 as its // in SCREEN_CAPTURE_FOLDER, beside the stills, through app/screen_capture.jai's jobs: the MP4 as its
@ -8,7 +9,8 @@
// recorded from in metadata.json, as for a still, and how it was recorded in capture.json. // recorded from in metadata.json, as for a still, and how it was recorded in capture.json.
// //
// While it records, a red border is drawn just outside the region and a toolbar beside it gives the // While it records, a red border is drawn just outside the region and a toolbar beside it gives the
// time, Pause, Stop and Discard (app/screen_record_win32.jai); both are kept out of the recording. // time, Pause, Stop and Discard (app/screen_record_win32.jai, app/screen_record_macos.jai); both are
// kept out of the recording.
// Time paused is left out of the video. With sound: what the computer plays and the microphone, as // Time paused is left out of the video. With sound: what the computer plays and the microphone, as
// [video] in the settings file says, as are the frame rate, the cursor and how the screen is taken. // [video] in the settings file says, as are the frame rate, the cursor and how the screen is taken.
// The toolbar's System sound and Mic buttons turn each on and off while recording, and are kept in // The toolbar's System sound and Mic buttons turn each on and off while recording, and are kept in
@ -19,7 +21,7 @@
// start (screen_record_leftovers_popup). PrintScreen while recording takes the screen under the cursor // start (screen_record_leftovers_popup). PrintScreen while recording takes the screen under the cursor
// at once, without the overlay, whose frozen screen would be in the recording. // at once, without the overlay, whose frozen screen would be in the recording.
// //
// Windows only, as modules/Screen_Record is. // Windows and macOS, as modules/Screen_Record is.
SCREEN_RECORD_PREFIX :: "recording-"; // the MP4s in the temporary folder: recording-2026-10-10_07-12-43.mp4 SCREEN_RECORD_PREFIX :: "recording-"; // the MP4s in the temporary folder: recording-2026-10-10_07-12-43.mp4
@ -32,7 +34,7 @@ screen_record: struct {
shot: Screen_Shot; // the region and the window it is of; no pixels shot: Screen_Shot; // the region and the window it is of; no pixels
date: string; // when it started, as CaptureDate; heap date: string; // when it started, as CaptureDate; heap
path: string; // the MP4 being written; heap path: string; // the MP4 being written; heap
#if OS == .WINDOWS recorder: Recorder; #if OS == .WINDOWS || OS == .MACOS recorder: Recorder;
} }
// From app_frame when the hotkey fired, outside the ImGui frame: starts recording, or stops. // From app_frame when the hotkey fired, outside the ImGui frame: starts recording, or stops.
@ -44,8 +46,9 @@ screen_record_toggle :: () {
// Every frame (from screen_capture_frame): a recording that stopped by itself is finished and kept. // Every frame (from screen_capture_frame): a recording that stopped by itself is finished and kept.
screen_record_frame :: () { screen_record_frame :: () {
if app.record_and_exit > 0 screen_record_flag_frame(); if app.record_and_exit > 0 screen_record_flag_frame();
#if OS == .WINDOWS { #if OS == .WINDOWS || OS == .MACOS {
if !screen_record.recording return; if !screen_record.recording return;
#if OS == .MACOS record_ui_tick();
failed, why := record_failed(*screen_record.recorder); failed, why := record_failed(*screen_record.recorder);
if failed { if failed {
toast_show(.Error, "The recording stopped: %", why); toast_show(.Error, "The recording stopped: %", why);
@ -122,7 +125,7 @@ screen_record_shutdown :: () {
// At startup: recordings a crash left in the temporary folder, to offer (screen_record_leftovers_popup). // At startup: recordings a crash left in the temporary folder, to offer (screen_record_leftovers_popup).
screen_record_startup :: () { screen_record_startup :: () {
#if OS == .WINDOWS { #if OS == .WINDOWS || OS == .MACOS {
screen_record.leftovers.allocator = context.default_allocator; screen_record.leftovers.allocator = context.default_allocator;
for file_list(screen_record_folder(),, temp) { for file_list(screen_record_folder(),, temp) {
name := path_filename(it); name := path_filename(it);
@ -193,17 +196,19 @@ screen_record_folder :: () -> string {
return tprint("%/%", platform_temporary_folder(), APP_FOLDER_NAME); return tprint("%/%", platform_temporary_folder(), APP_FOLDER_NAME);
} }
#if OS == .WINDOWS { #if OS == .WINDOWS || OS == .MACOS {
// immediate: the screen under the cursor, without the overlay (-record). // immediate: the screen under the cursor, without the overlay (-record).
screen_record_start :: (immediate := false) { screen_record_start :: (immediate := false) {
if app_exit_shutdown_started() return; if app_exit_shutdown_started() return;
date := import_calendar_text(to_calendar(current_time_consensus(), .LOCAL)); date := import_calendar_text(to_calendar(current_time_consensus(), .LOCAL));
shot, ok, error := platform_screen_capture_select(video = true, immediate = immediate); shot, ok, error := platform_screen_capture_select(video = true, immediate = immediate);
r := app.record_region; #if OS == .WINDOWS {
if ok && immediate && r.right > r.left { // -record-region r := app.record_region;
shot.left, shot.top, shot.width, shot.height = r.left, r.top, r.right - r.left, r.bottom - r.top; if ok && immediate && r.right > r.left { // -record-region
shot.whole_screen = false; shot.left, shot.top, shot.width, shot.height = r.left, r.top, r.right - r.left, r.bottom - r.top;
shot.whole_screen = false;
}
} }
if !ok { if !ok {
if error { log_error("[record] %.", error); toast_show(.Error, "Screen recording: %", error); } if error { log_error("[record] %.", error); toast_show(.Error, "Screen recording: %", error); }
@ -220,18 +225,28 @@ screen_record_start :: (immediate := false) {
options: Record_Options; options: Record_Options;
options.path = path; options.path = path;
options.region = .{shot.left, shot.top, shot.left + shot.width, shot.top + shot.height};
options.fps = cast(s32) app_user_settings.video_fps; options.fps = cast(s32) app_user_settings.video_fps;
options.cursor = app_user_settings.video_cursor; options.cursor = app_user_settings.video_cursor;
options.system_audio = app_user_settings.video_system_audio; options.system_audio = app_user_settings.video_system_audio;
options.microphone = app_user_settings.video_microphone; options.microphone = app_user_settings.video_microphone;
options.audio_track = true; // so the toolbar can turn a source on mid-recording options.audio_track = true; // so the toolbar can turn a source on mid-recording
if app_user_settings.video_capture == { #if OS == .MACOS {
case "dxgi"; options.capture = .DXGI; // The display's region, and the border and toolbar first: ScreenCaptureKit leaves them out of the
case "wgc"; options.capture = .WGC; // recording by their window numbers.
options.display_id = shot.display_id;
options.x, options.y, options.w, options.h = shot.local_x, shot.local_y, shot.local_width, shot.local_height;
options.width, options.height = shot.width, shot.height;
options.exclude_windows = record_ui_open(shot, shot.width & cast(s32) ~1, shot.height & cast(s32) ~1);
} else {
options.region = .{shot.left, shot.top, shot.left + shot.width, shot.top + shot.height};
if app_user_settings.video_capture == {
case "dxgi"; options.capture = .DXGI;
case "wgc"; options.capture = .WGC;
}
} }
started, start_error := record_start(*screen_record.recorder, options); started, start_error := record_start(*screen_record.recorder, options);
if !started { if !started {
#if OS == .MACOS record_ui_close();
log_error("[record] Could not record % x % at %, %: %", shot.width, shot.height, shot.left, shot.top, start_error); log_error("[record] Could not record % x % at %, %: %", shot.width, shot.height, shot.left, shot.top, start_error);
toast_show(.Error, "Could not record: %", start_error); toast_show(.Error, "Could not record: %", start_error);
screen_shot_free(*shot); screen_shot_free(*shot);
@ -253,7 +268,7 @@ screen_record_start :: (immediate := false) {
} else { } else {
toast_show(.Success, "Recording % x %: % stops", stats.width, stats.height, screen_record_hotkey_text()); toast_show(.Success, "Recording % x %: % stops", stats.width, stats.height, screen_record_hotkey_text());
} }
record_ui_open(options.region, shot.whole_screen, stats.width, stats.height); #if OS == .WINDOWS record_ui_open(options.region, shot.whole_screen, stats.width, stats.height);
} }
// A sound button's state, into [video] for the next recording. // A sound button's state, into [video] for the next recording.
@ -411,13 +426,13 @@ screen_record_hotkey_text :: () -> string {
screen_record_start :: (immediate := false) { screen_record_start :: (immediate := false) {
toast_show(.Error, "Screen recording is not available on this system yet"); toast_show(.Error, "Screen recording is not available on this system yet");
if app.record_and_exit > 0 screen_record_saved("", "screen recording is Windows only"); if app.record_and_exit > 0 screen_record_saved("", "screen recording is on Windows and macOS only");
} }
screen_record_stop :: (shutdown := false) {} screen_record_stop :: (shutdown := false) {}
screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string { screen_record_write :: (job: *Screen_Capture_Job) -> path: string, error: string {
return "", "screen recording is Windows only"; return "", "screen recording is on Windows and macOS only";
} }
} }

365
src/app/screen_record_macos.jai Normal file
View File

@ -0,0 +1,365 @@
// While recording (app/screen_record.jai), on macOS: a thin red border just outside the region, and a
// toolbar beside it with the time, the size, frames dropped, and System sound, Mic, Pause, Stop and
// Discard, as app/screen_record_win32.jai has on Windows. Both are non-activating panels above other
// windows, on every Space and over full-screen apps, so they work while mcap is hidden in the menu
// bar and a click on the toolbar leaves the window being recorded in front. Both are kept out of the
// recording: their window numbers go to ScreenCaptureKit's filter (record_ui_open returns them), so
// they are made before the recording starts. The border is outside the region anyway.
//
// The border lets clicks through. A toolbar button only asks (record_ui.asked); screen_record_frame
// does it. Discard asks first, in the toolbar itself. Hover and the time are kept up from
// screen_record_frame (record_ui_tick), as the panels get no events while the mouse only passes.
Record_UI_Ask :: enum u8 { NONE; SYSTEM_SOUND; MICROPHONE; PAUSE; STOP; DISCARD; }
record_ui: struct {
border, toolbar: *NSWindow;
toolbar_view: *NSView;
toolbar_frame: CGRect; // in points from the main screen's top left, for the hover
asked: Record_UI_Ask; // a button pressed, for screen_record_frame
confirming: bool; // Discard pressed once: "Discard it? Yes No"
hovered: s32 = -1; // the button under the mouse
buttons: [5] Record_UI_Button;
button_count: s32;
started: Apollo_Time;
drawn: Apollo_Time; // the toolbar last drawn
size_text: string; // "1920 × 1080"; heap
}
Record_UI_Button :: struct { rect: CGRect; label: string; ask: Record_UI_Ask; confirm: bool; }
// Shows the border around the shot's region and the toolbar beside it; their window numbers (temp),
// to keep them out of the recording. width and height: what is recorded, in pixels.
record_ui_open :: (shot: Screen_Shot, width: s32, height: s32) -> [] u32 {
record_ui_close();
init_objective_c();
init_core_graphics_runtime_constants();
record_ui.started = current_time_monotonic();
record_ui.drawn = record_ui.started;
record_ui.asked = .NONE;
record_ui.confirming = false;
record_ui.hovered = -1;
free(record_ui.size_text,, context.default_allocator);
record_ui.size_text = sprint("% × %", width, height,, context.default_allocator);
region := CGRect.{.{shot.screen_left + shot.local_x, shot.screen_top + shot.local_y}, .{shot.local_width, shot.local_height}};
screen := CGRect.{.{shot.screen_left, shot.screen_top}, .{shot.screen_width, shot.screen_height}};
// Below the region, at its right; else above; else inside its top right (kept out of the recording).
toolbar_width, toolbar_height := record_toolbar_size();
x := region.origin.x + region.size.width - toolbar_width;
y := region.origin.y + region.size.height + BORDER + GAP;
if y + toolbar_height > screen.origin.y + screen.size.height y = region.origin.y - BORDER - GAP - toolbar_height;
if y < screen.origin.y || shot.whole_screen {
y = region.origin.y + GAP;
x = region.origin.x + region.size.width - toolbar_width - GAP;
}
x = clamp(x, screen.origin.x, screen.origin.x + screen.size.width - toolbar_width);
record_ui.toolbar_frame = .{.{x, y}, .{toolbar_width, toolbar_height}};
record_ui.toolbar, record_ui.toolbar_view = record_ui_panel(record_ui.toolbar_frame, TOOLBAR, clicks = true);
// The border: BORDER points wide just outside the region. Not for a whole screen, where it would be
// on the screen beside it.
if !shot.whole_screen {
outer := CGRect.{.{region.origin.x - BORDER, region.origin.y - BORDER}, .{region.size.width + 2 * BORDER, region.size.height + 2 * BORDER}};
record_ui.border = record_ui_panel(outer, BORDER_VIEW, clicks = false);
}
numbers: [..] u32;
numbers.allocator = temp;
if array_find(app.args, "-record-ui-visible") return numbers; // to look at them in a recording
for window: *NSWindow.[record_ui.toolbar, record_ui.border] if window {
func :: #type (*void, Selector) -> s64 #c_call;
array_add(*numbers, cast(u32) (cast(func) objc_msgSend)(window, selector("windowNumber")));
}
return numbers;
}
record_ui_close :: () {
for window: *NSWindow.[record_ui.toolbar, record_ui.border] if window {
NSWindow.orderOut(window, null);
NSWindow.close(window);
release(window);
}
record_ui.toolbar, record_ui.border, record_ui.toolbar_view = null, null, null;
}
// From another thread (the shutdown's): hidden on the main thread, the only one AppKit takes.
record_ui_hide :: () {
hide :: (data: *void) #c_call {
new_context: #Context;
push_context new_context {
for window: *NSWindow.[record_ui.toolbar, record_ui.border] if window NSWindow.orderOut(window, null);
}
}
dispatch_async_f(*_dispatch_main_q, null, hide);
}
// The button pressed since the last call, once.
record_ui_take_ask :: () -> Record_UI_Ask {
ask := record_ui.asked;
record_ui.asked = .NONE;
return ask;
}
record_ui_refresh :: () {
set_needs_display :: #type (*void, Selector, BOOL) -> void #c_call;
send := cast(set_needs_display) objc_msgSend;
if record_ui.toolbar_view send(record_ui.toolbar_view, selector("setNeedsDisplay:"), YES);
}
// Each frame while recording: the button under the mouse lit, and the time moved on four times a second.
record_ui_tick :: () {
if !record_ui.toolbar_view return;
location := NSEvent.mouseLocation();
main_height := main_screen_height();
f := record_ui.toolbar_frame;
point := Vector2.{xx (location.x - f.origin.x), xx (main_height - location.y - f.origin.y)};
hovered := record_toolbar_hit(point);
now := current_time_monotonic();
if hovered != record_ui.hovered || to_float64_seconds(now - record_ui.drawn) >= 0.25 {
record_ui.hovered = hovered;
record_ui.drawn = now;
record_ui_refresh();
}
}
#scope_file
BORDER : float64 : 3.0; // points
GAP : float64 : 6.0;
TOOLBAR :: 1; // the view's kind, in its extra bytes
BORDER_VIEW :: 2;
RED :: Vector4.{0.91, 0.157, 0.157, 1}; // as the overlay's video mode
DIM_RED:: Vector4.{0.38, 0.125, 0.125, 1};
GREY :: Vector4.{0.565, 0.565, 0.565, 1};
AMBER :: Vector4.{1, 0.706, 0, 1};
BACK :: Vector4.{0.125, 0.125, 0.125, 0.96};
BUTTON :: Vector4.{0.22, 0.22, 0.22, 1};
HOVER :: Vector4.{0.314, 0.314, 0.314, 1};
TEXT :: Vector4.{0.94, 0.94, 0.94, 1};
PADDING : float64 : 10.0;
SPACE : float64 : 8.0;
HEIGHT : float64 : 34.0;
DOT : float64 : 12.0;
main_screen_height :: () -> float64 {
screens := objc_msgSend_typed(objc_getClass("NSScreen"), selector("screens"));
func_at :: #type (*void, Selector, u64) -> *void #c_call;
first := (cast(func_at) objc_msgSend)(screens, selector("objectAtIndex:"), 0);
func_frame :: #type (*void, Selector) -> NSRect #c_call;
return (cast(func_frame) objc_msgSend)(first, selector("frame")).size.height;
}
// A borderless, non-activating panel at frame (points from the main screen's top left) above other
// windows, its content a McapRecordView of kind; letting clicks through unless clicks.
record_ui_panel :: (frame: CGRect, kind: s64, clicks: bool) -> *NSWindow, *NSView {
view_class := record_view_class();
if !view_class return null, null;
main_height := main_screen_height();
NSWindowStyleMaskNonactivatingPanel :: 1 << 7;
window := NSWindow.initWithContentRect(cast(*NSWindow) objc_alloc(objc_getClass("NSPanel")),
NSMakeRect(frame.origin.x, main_height - frame.origin.y - frame.size.height, frame.size.width, frame.size.height),
NSWindowStyleMaskBorderless | NSWindowStyleMaskNonactivatingPanel, NSBackingStoreBuffered, NO);
if !window return null, null;
set_bool :: (object: *void, name: string, value: bool) {
func :: #type (*void, Selector, BOOL) -> void #c_call;
(cast(func) objc_msgSend)(object, selector(name), ifx value then YES else NO);
}
NSWindow.setReleasedWhenClosed(window, NO);
set_bool(window, "setOpaque:", false);
set_bool(window, "setHasShadow:", kind == TOOLBAR);
set_bool(window, "setHidesOnDeactivate:", false);
set_bool(window, "setIgnoresMouseEvents:", !clicks);
set_bool(window, "setBecomesKeyOnlyIfNeeded:", true);
NSWindow.setBackgroundColor(window, objc_msgSend_typed(objc_getClass("NSColor"), selector("clearColor")));
NSStatusWindowLevel :: 25;
NSWindow.setLevel(window, NSStatusWindowLevel);
NSWindow.setCollectionBehavior(window, NSWindowCollectionBehaviorCanJoinAllSpaces | NSWindowCollectionBehaviorStationary |
NSWindowCollectionBehaviorFullScreenAuxiliary | NSWindowCollectionBehaviorIgnoresCycle);
func_init :: #type (*void, Selector, NSRect) -> *NSView #c_call;
view := (cast(func_init) objc_msgSend)(objc_alloc(view_class), selector("initWithFrame:"), NSMakeRect(0, 0, frame.size.width, frame.size.height));
(cast(*s64) object_getIndexedIvars(view)).* = kind;
NSWindow.setContentView(window, view);
release(view);
objc_msgSend_typed(window, selector("orderFrontRegardless"));
return window, view;
}
// McapRecordView: draws the border or the toolbar (its kind in its extra bytes), y going down, and
// takes the toolbar's clicks without making mcap active.
record_view_class :: () -> Class {
if view_class return view_class;
view_class = objc_getClass("McapRecordView");
if view_class return view_class;
view_class = objc_allocateClassPair(objc_getClass("NSView"), "McapRecordView", size_of(s64));
if !view_class return null;
draw_rect :: (self: id, _sel: Selector, dirty: NSRect) #c_call {
push_context record_context {
kind := (cast(*s64) object_getIndexedIvars(self)).*;
context_object := objc_msgSend_typed(objc_getClass("NSGraphicsContext"), selector("currentContext"));
cg := ifx context_object then cast(*void) objc_msgSend_typed(context_object, selector("CGContext")) else null;
if !cg return;
func_bounds :: #type (*void, Selector) -> NSRect #c_call;
bounds := (cast(func_bounds) objc_msgSend)(self, selector("bounds"));
if kind == BORDER_VIEW draw_border(cg, bounds.size.width, bounds.size.height);
else draw_toolbar(cg, bounds.size.width, bounds.size.height);
}
}
mouse_down :: (self: id, _sel: Selector, event: *NSEvent) #c_call {
push_context record_context {
func_location :: #type (*void, Selector) -> NSPoint #c_call;
location := (cast(func_location) objc_msgSend)(event, selector("locationInWindow"));
height := record_ui.toolbar_frame.size.height;
hit := record_toolbar_hit(.{xx location.x, xx (height - location.y)});
if hit >= 0 {
button := record_ui.buttons[hit];
if button.confirm record_ui.confirming = !record_ui.confirming;
else if button.ask != .NONE { record_ui.asked = button.ask; record_ui.confirming = false; }
record_ui_refresh();
}
}
}
yes :: (self: id, _sel: Selector) -> BOOL #c_call { return YES; }
yes_for_event :: (self: id, _sel: Selector, event: *NSEvent) -> BOOL #c_call { return YES; }
class_addMethod(view_class, selector("drawRect:"), xx draw_rect, "v@:{CGRect={CGPoint=dd}{CGSize=dd}}");
class_addMethod(view_class, selector("mouseDown:"), xx mouse_down, "v@:@");
class_addMethod(view_class, selector("isFlipped"), xx yes, "c@:");
class_addMethod(view_class, selector("acceptsFirstMouse:"), xx yes_for_event, "c@:@");
objc_registerClassPair(view_class);
record_context = context;
return view_class;
}
view_class: Class;
record_context: #Context; // AppKit calls the view on the main thread, during SDL's event pump
fill :: (cg: *void, r: CGRect, color: Vector4) {
CGContextSetRGBFillColor(cg, color.x, color.y, color.z, color.w);
CGContextFillRect(cg, r);
}
draw_border :: (cg: *void, width: float64, height: float64) {
fill(cg, .{.{0, 0}, .{width, BORDER}}, RED);
fill(cg, .{.{0, height - BORDER}, .{width, BORDER}}, RED);
fill(cg, .{.{0, BORDER}, .{BORDER, height - 2 * BORDER}}, RED);
fill(cg, .{.{width - BORDER, BORDER}, .{BORDER, height - 2 * BORDER}}, RED);
}
draw_toolbar :: (cg: *void, width: float64, height: float64) {
fill(cg, .{.{0, 0}, .{width, height}}, BACK);
paused := record_paused(*screen_record.recorder);
// The dot: red while recording, blinking each half second; grey while paused.
blink := (cast(s64) (to_float64_seconds(current_time_monotonic() - record_ui.started) * 2)) % 2 == 0;
dot := ifx paused then GREY else ifx blink then RED else DIM_RED;
CGContextSetRGBFillColor(cg, dot.x, dot.y, dot.z, dot.w);
CGContextFillEllipseInRect(cg, .{.{PADDING, (height - DOT) / 2}, .{DOT, DOT}});
status, dropped := record_toolbar_texts();
x := PADDING + DOT + SPACE;
draw_text(status, x, height, TEXT);
x += text_size(status).width + SPACE;
if dropped draw_text(dropped, x, height, AMBER);
// The buttons, from the right.
buttons := record_toolbar_buttons();
button_height := height - 10;
right := width - PADDING;
for < buttons {
label_width := text_size(it.label).width;
r := CGRect.{.{right - label_width - 2 * PADDING, (height - button_height) / 2}, .{label_width + 2 * PADDING, button_height}};
record_ui.buttons[it_index].rect = r;
lit := (it.ask == .SYSTEM_SOUND && record_audio_on(*screen_record.recorder, .SYSTEM)) || (it.ask == .MICROPHONE && record_audio_on(*screen_record.recorder, .MICROPHONE));
fill(cg, r, ifx lit then RED else ifx it_index == record_ui.hovered then HOVER else BUTTON);
draw_text(it.label, r.origin.x + PADDING, height, ifx it.ask == .DISCARD then AMBER else TEXT);
right = r.origin.x - SPACE;
}
if record_ui.confirming draw_text("Discard it?", right - text_size("Discard it?").width, height, AMBER);
}
// What the toolbar shows now, left to right: the status, then what went wrong.
record_toolbar_texts :: () -> status: string, dropped: string {
stats := record_stats(*screen_record.recorder);
paused := record_paused(*screen_record.recorder);
seconds := max(to_float64_seconds(current_time_monotonic() - record_ui.started) - stats.paused_seconds, 0.0);
whole := cast(s64) seconds;
two :: (n: s64) -> string { return tprint("%", formatInt(n, minimum_digits = 2)); }
time := ifx whole >= 3600 then tprint("%:%:%", whole / 3600, two(whole / 60 % 60), two(whole % 60)) else tprint("%:%", two(whole / 60), two(whole % 60));
status := tprint("% % %", ifx paused then "PAUSED" else "REC", time, record_ui.size_text);
dropped := ifx stats.dropped then tprint("% dropped", stats.dropped) else "";
return status, dropped;
}
record_toolbar_buttons :: () -> [] Record_UI_Button {
if record_ui.confirming {
record_ui.buttons[0] = .{label = "Yes, discard", ask = .DISCARD};
record_ui.buttons[1] = .{label = "No", confirm = true};
record_ui.button_count = 2;
} else {
recorder := *screen_record.recorder;
record_ui.buttons[0] = .{label = ifx record_audio_on(recorder, .SYSTEM) then "System sound: on" else "System sound: off", ask = .SYSTEM_SOUND};
record_ui.buttons[1] = .{label = ifx record_audio_on(recorder, .MICROPHONE) then "Mic: on" else "Mic: off", ask = .MICROPHONE};
record_ui.buttons[2] = .{label = ifx record_paused(recorder) then "Resume" else "Pause", ask = .PAUSE};
record_ui.buttons[3] = .{label = "Stop", ask = .STOP};
record_ui.buttons[4] = .{label = "Discard", confirm = true};
record_ui.button_count = 5;
}
return array_view(record_ui.buttons, 0, record_ui.button_count);
}
// The toolbar's size, for the longest it is likely to be.
record_toolbar_size :: () -> width: float64, height: float64 {
status_width := text_size(tprint("PAUSED 00:00:00 %", record_ui.size_text)).width;
dropped_width := text_size("999 dropped").width;
buttons_width: float64 = 4 * SPACE + 5 * 2 * PADDING;
for string.["System sound: off", "Mic: off", "Resume", "Stop", "Yes, discard"] buttons_width += text_size(it).width;
return PADDING + DOT + SPACE + status_width + SPACE + dropped_width + SPACE + buttons_width + PADDING, HEIGHT;
}
record_toolbar_hit :: (p: Vector2) -> s32 {
for array_view(record_ui.buttons, 0, record_ui.button_count) {
r := it.rect;
if p.x >= r.origin.x && p.x < r.origin.x + r.size.width && p.y >= r.origin.y && p.y < r.origin.y + r.size.height return cast(s32) it_index;
}
return -1;
}
text_attributes :: (color: Vector4) -> *NSObject {
func_font :: #type (*void, Selector, CGFloat, CGFloat) -> *NSObject #c_call;
font := (cast(func_font) objc_msgSend)(objc_getClass("NSFont"), selector("monospacedDigitSystemFontOfSize:weight:"), 13, 0.3);
func_color :: #type (*void, Selector, CGFloat, CGFloat, CGFloat, CGFloat) -> *NSObject #c_call;
ns_color := (cast(func_color) objc_msgSend)(objc_getClass("NSColor"), selector("colorWithSRGBRed:green:blue:alpha:"), color.x, color.y, color.z, color.w);
keys := *void.[NSFontAttributeName, NSForegroundColorAttributeName];
values := *void.[font, ns_color];
func_dictionary :: #type (*void, Selector, **void, **void, NSUInteger) -> *NSObject #c_call;
return (cast(func_dictionary) objc_msgSend)(objc_getClass("NSDictionary"), selector("dictionaryWithObjects:forKeys:count:"), values.data, keys.data, 2);
}
text_size :: (text: string) -> NSSize {
func :: #type (*void, Selector, *NSObject) -> NSSize #c_call;
return (cast(func) objc_msgSend)(NSString.getTempString(text), selector("sizeWithAttributes:"), text_attributes(TEXT));
}
// Text at x, centred in height.
draw_text :: (text: string, x: float64, height: float64, color: Vector4) {
size := text_size(text);
func :: #type (*void, Selector, NSPoint, *NSObject) -> void #c_call;
(cast(func) objc_msgSend)(NSString.getTempString(text), selector("drawAtPoint:withAttributes:"), NSMakePoint(x, (height - size.height) / 2), text_attributes(color));
}
CGRect :: struct { origin: struct { x, y: float64; }; size: struct { width, height: float64; }; }
CoreGraphics_Library :: #library,system "CoreGraphics";
CGContextSetRGBFillColor :: (c: *void, r: float64, g: float64, b: float64, a: float64) #foreign CoreGraphics_Library;
CGContextFillRect :: (c: *void, rect: CGRect) #foreign CoreGraphics_Library;
CGContextFillEllipseInRect :: (c: *void, rect: CGRect) #foreign CoreGraphics_Library;
AppKit_Library :: #library,system "AppKit";
NSFontAttributeName: *NSString #elsewhere AppKit_Library;
NSForegroundColorAttributeName: *NSString #elsewhere AppKit_Library;
libSystem_Library :: #library,system "libSystem";
_dispatch_main_q: u8 #elsewhere libSystem_Library;
dispatch_async_f :: (queue: *void, data: *void, work: (data: *void) #c_call) #foreign libSystem_Library;
#import "Objective_C";
#import "Objective_C/AppKit";

2
src/ui/imgui_keyboard.jai
View File

@ -350,7 +350,7 @@ GLOBAL_HOTKEY_ACTION_COUNT :: 3; // sizes the platforms' registration a
global_hotkey_actions_build :: (out: *[..] Hotkey_Action) { global_hotkey_actions_build :: (out: *[..] Hotkey_Action) {
array_add(out, .{"bring_to_foreground", *bring_to_foreground_hotkey, false}); array_add(out, .{"bring_to_foreground", *bring_to_foreground_hotkey, false});
array_add(out, .{"capture_region", *capture_region_hotkey, OS == .LINUX}); // app/screen_capture.jai: Windows and macOS array_add(out, .{"capture_region", *capture_region_hotkey, OS == .LINUX}); // app/screen_capture.jai: Windows and macOS
array_add(out, .{"record_region", *record_region_hotkey, true}); // app/screen_record.jai array_add(out, .{"record_region", *record_region_hotkey, OS == .LINUX}); // app/screen_record.jai: Windows and macOS
} }
global_hotkey_actions: [..] Hotkey_Action; global_hotkey_actions: [..] Hotkey_Action;

4
src/ui/imgui_theme_registry.jai
View File

@ -736,8 +736,8 @@ app_save_user_settings_file :: () -> bool {
append(*builder, "# starts; -library FOLDER on the command line overrides it.\n"); append(*builder, "# starts; -library FOLDER on the command line overrides it.\n");
print_to_builder(*builder, "folder: %\n\n", settings_quote_if_needed(ifx app_user_settings.library_folder.count then app_user_settings.library_folder else LIBRARY_FOLDER_DEFAULT)); print_to_builder(*builder, "folder: %\n\n", settings_quote_if_needed(ifx app_user_settings.library_folder.count then app_user_settings.library_folder else LIBRARY_FOLDER_DEFAULT));
append(*builder, "[video]\n"); append(*builder, "[video]\n");
append(*builder, "# Screen recording (Ctrl+PrintScreen; Windows). fps: frames a second at most, written only when the\n"); append(*builder, "# Screen recording (Ctrl+PrintScreen; Opt+Cmd+Shift+2 on macOS). fps: frames a second at most, written only when the\n");
append(*builder, "# screen changes. cursor: true or false. capture: auto (Desktop Duplication, else\n"); append(*builder, "# screen changes. cursor: true or false. capture (Windows): auto (Desktop Duplication, else\n");
append(*builder, "# Windows.Graphics.Capture), dxgi or wgc. system_audio: what the computer plays; microphone: the\n"); append(*builder, "# Windows.Graphics.Capture), dxgi or wgc. system_audio: what the computer plays; microphone: the\n");
append(*builder, "# default input device. Each true or false.\n"); append(*builder, "# default input device. Each true or false.\n");
print_to_builder(*builder, "fps: %\n", app_user_settings.video_fps); print_to_builder(*builder, "fps: %\n", app_user_settings.video_fps);