MusaMahmood/mcap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

[0.1.4] Video capture phase 2: Windows.Graphics.Capture, for when Desktop Duplication cannot run

Browse Source 
- modules/Screen_Record/wgc_source.jai: the screen through Windows.Graphics.Capture, with
  hand-written bindings for its WinRT interfaces (capture item interop, item, free-threaded frame
  pool, frame, session 1-3, IDirect3DDxgiInterfaceAccess, IClosable). A small COM handler on
  FrameArrived sets an event per frame; when Windows refuses it the pool is polled each
  millisecond. The cursor is drawn by Windows; the yellow border is turned off where the system
  allows (Windows 11). A screen that changes size makes the pool again. WinRT's own calls are
  loaded from combase.dll at run time: runtimeobject.lib names no DLL for #library.
- capture_gpu.jai: what both sources share (the device on the screen's adapter, the region copy,
  the staging ring), so each is only "the next image's region into a texture".
- Record_Options.capture: AUTO (Desktop Duplication, else Windows.Graphics.Capture when it says
  fall_back), DXGI or WGC. Sources open on the capture thread, as WinRT objects belong to their
  apartment; record_start waits for it to say how that went. Record_Options.testing simulates the
  two-GPU failure, forces polling, or keeps the border, for record-probe.
- record-probe: -capture auto|dxgi|wgc, -simulate-unsupported, -poll, -border-test with
  -keep-border as its control, -cursor-test; DPI awareness set already is no longer reported as
  a failure.
- On the RTX 5080 laptop: auto falls back on the simulated failure; WGC records 4K at 60 fps with
  nothing dropped; its cursor and the one cursor.jai draws for duplication are the same; the
  border is off for an unpackaged app (0 yellow edge pixels, 14,214 with it kept);
  WDA_EXCLUDEFROMCAPTURE is respected; polling holds 30 fps. The plan records the results.
- README: Screen_Record's line, which held a carriage return in place of "\r" since 0.1.3.
This commit is contained in:
Musa Mahmood 2026-10-10 07:03:47 +01:00
parent 06a2aacc9c
commit 9953ba6f1f
9 changed files with 830 additions and 171 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@ -357,9 +357,11 @@ converted copy.
(`video.jai`), and how many frames a GIF has. `bin\image-probe FILE...` (`jai first.jai - (`video.jai`), and how many frames a GIF has. `bin\image-probe FILE...` (`jai first.jai -
image_probe`) shows what it makes of files. image_probe`) shows what it makes of files.
- `modules/Screen_Record`: a region of one screen recorded to H.264 in an MP4, with DXGI Desktop - `modules/Screen_Record`: a region of one screen recorded to H.264 in an MP4, with DXGI Desktop
Duplication and FFmpeg (NVENC, else AMF, QSV, Media Foundation or libx264), for screen recording Duplication, or Windows.Graphics.Capture where duplication cannot run (two GPUs, a rotated screen),
(`video-capture-plan.html`). `bin ecord-probe` (`jai first.jai - record_probe`) records with it and FFmpeg (NVENC, else AMF, QSV, Media Foundation or libx264), for screen recording
and says how it went; `-list` shows the screens, `-check FILE` reads a recording back. (`video-capture-plan.html`). `bin\record-probe` (`jai first.jai - record_probe`) records with it
and says how it went; `-list` shows the screens, `-capture auto|dxgi|wgc` picks the way, `-check
FILE` reads a recording back.
- `modules/Snagx`: `.snagx` to `.mcap`, copied from SnagitConverter. - `modules/Snagx`: `.snagx` to `.mcap`, copied from SnagitConverter.
- `modules/Capture_File`: reading and writing `.mcap` files, with a video's original left on disk. - `modules/Capture_File`: reading and writing `.mcap` files, with a video's original left on disk.
- `modules/Segment`: panoptic segmentation with Mask2Former, run with onnxruntime (see its README and - `modules/Segment`: panoptic segmentation with Mask2Former, run with onnxruntime (see its README and

2
check_targets.jai
View File

@ -29,7 +29,7 @@ APP_NAME :: "mcap";
APP_FOLDER_NAME :: "mcap"; APP_FOLDER_NAME :: "mcap";
GIT_BRANCH :: "check"; GIT_BRANCH :: "check";
GIT_HASH :: "-"; GIT_HASH :: "-";
VERSION :: "0.1.3"; VERSION :: "0.1.4";
BUILD_DATE :: "-"; BUILD_DATE :: "-";
DEBUG :: true; DEBUG :: true;
COMPILE_TESTS :: true; COMPILE_TESTS :: true;

2
first.jai
View File

@ -8,7 +8,7 @@ APP_NAME :: "mcap";
APP_FOLDER_NAME :: "mcap"; APP_FOLDER_NAME :: "mcap";
APP_BUNDLE_IDENTIFIER :: "com.example.mcap"; APP_BUNDLE_IDENTIFIER :: "com.example.mcap";
VERSION :: "0.1.3"; VERSION :: "0.1.4";
// Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated // Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated
// builds never write over the ones you run by hand. // builds never write over the ones you run by hand.
build_directory := "bin"; build_directory := "bin";

105
modules/Screen_Record/capture_gpu.jai Normal file
View File

@ -0,0 +1,105 @@
// What both ways of taking the screen share (dxgi_source.jai, wgc_source.jai): a D3D11 device of
// their own on the adapter the screen is attached to, the region as last seen, copied out of each
// new image on the GPU, and the staging textures it is read back through.
Capture_Gpu :: struct {
adapter: *IDXGIAdapter1;
device: *ID3D11Device;
device_context: *ID3D11DeviceContext;
box: D3D11_BOX; // the region, in the screen's pixels
latest: *ID3D11Texture2D; // the region as last seen
staging: [RECORD_STAGING] *ID3D11Texture2D;
copy_ticks: s64; // spent taking the region out of images (QueryPerformanceCounter ticks)
}
RECORD_STAGING :: 3; // staging textures in turn: the one read is two frames behind the one copied
// What a source's next() found.
Capture_Next :: enum u8 { NONE; IMAGE; POINTER; IMAGE_AND_POINTER; LOST; FAILED; }
// The device on the screen's adapter, and the textures for a region (in the screen's pixels).
gpu_open :: (g: *Capture_Gpu, screen: Record_Screen, region: RECT) -> ok: bool, error: string {
factory: *IDXGIFactory1;
hr := CreateDXGIFactory1(*uid(IDXGIFactory1_UUID), xx *factory);
if FAILED(hr) return false, hresult_text("CreateDXGIFactory1", hr);
defer IUnknown_Release(factory);
hr = IDXGIFactory1_EnumAdapters1(factory, screen.adapter_index, *g.adapter);
if FAILED(hr) return false, hresult_text("the screen's adapter is gone", hr);
levels := D3D_FEATURE_LEVEL.[._11_1, ._11_0, ._10_1, ._10_0];
level: D3D_FEATURE_LEVEL;
hr = D3D11CreateDevice(g.adapter, .UNKNOWN, null, .BGRA_SUPPORT, levels.data, levels.count, D3D11_SDK_VERSION, *g.device, *level, *g.device_context);
if FAILED(hr) return false, hresult_text("D3D11CreateDevice on the screen's adapter", hr);
g.box = .{left = xx region.left, top = xx region.top, right = xx region.right, bottom = xx region.bottom, front = 0, back = 1};
desc: D3D11_TEXTURE2D_DESC;
desc.Width = cast(u32) (region.right - region.left);
desc.Height = cast(u32) (region.bottom - region.top);
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = .B8G8R8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.Usage = .DEFAULT;
hr = ID3D11Device_CreateTexture2D(g.device, *desc, null, *g.latest);
if FAILED(hr) return false, hresult_text("a texture for the region", hr);
desc.Usage = .STAGING;
desc.CPUAccessFlags = .READ;
for * g.staging {
hr = ID3D11Device_CreateTexture2D(g.device, *desc, null, it);
if FAILED(hr) return false, hresult_text("a staging texture", hr);
}
return true, "";
}
// The region of a whole screen's image into g.latest.
gpu_copy_region :: (g: *Capture_Gpu, image: *ID3D11Texture2D) {
start := ticks_now();
ID3D11DeviceContext_CopySubresourceRegion(g.device_context, g.latest, 0, 0, 0, 0, image, 0, *g.box);
g.copy_ticks += ticks_now() - start;
}
gpu_close :: (g: *Capture_Gpu) {
for * g.staging com_release(it);
com_release(*g.latest);
com_release(*g.device_context);
com_release(*g.device);
com_release(*g.adapter);
}
#scope_module
com_release :: (p: *$T) {
if p.* { IUnknown_Release(cast(*IUnknown) p.*); p.* = null; }
}
ticks_now :: () -> s64 {
t: LARGE_INTEGER;
QueryPerformanceCounter(*t);
return t.QuadPart;
}
hresult_is :: (hr: HRESULT, code: u32) -> bool {
return cast,no_check(u32) hr == code;
}
// "what failed: 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", temp.
hresult_text :: (what: string, hr: HRESULT) -> string {
code := cast,no_check(u32) hr;
name := "";
if code == {
case DXGI_ERROR_UNSUPPORTED; name = " (DXGI_ERROR_UNSUPPORTED)";
case DXGI_ERROR_NOT_CURRENTLY_AVAILABLE; name = " (DXGI_ERROR_NOT_CURRENTLY_AVAILABLE)";
case DXGI_ERROR_ACCESS_LOST; name = " (DXGI_ERROR_ACCESS_LOST)";
case 0x80070005; name = " (E_ACCESSDENIED)";
case 0x80004002; name = " (E_NOINTERFACE)";
case 0x80070057; name = " (E_INVALIDARG)";
case 0x80040154; name = " (REGDB_E_CLASSNOTREG)";
case 0x80004001; name = " (E_NOTIMPL)";
}
return tprint("%: 0x%%", what, formatInt(code, base = 16, minimum_digits = 8), name);
}
wide_text :: (wide: *s16) -> string {
text, ok := wide_to_utf8(cast(*u16) wide,, temp);
return ifx ok then text else "";
}

124
modules/Screen_Record/dxgi_source.jai
View File

@ -1,37 +1,29 @@
// DXGI Desktop Duplication: the screens as the GPU composes them, a frame each time one changes. // DXGI Desktop Duplication: the screens as the GPU composes them, a frame each time one changes,
// with the cursor apart (cursor.jai draws it in).
// //
// It runs on a D3D11 device of its own, made on the adapter the screen is attached to: duplicating // It runs on a device made on the adapter the screen is attached to (capture_gpu.jai): duplicating
// with a device on another adapter fails. On a laptop with two GPUs, where the screen hangs off one // with a device on another adapter fails. On a laptop with two GPUs, where the screen hangs off one
// and the process may run on the other, DuplicateOutput1 can fail with DXGI_ERROR_UNSUPPORTED all the // and the process may run on the other, DuplicateOutput1 can fail with DXGI_ERROR_UNSUPPORTED all the
// same; that, too many duplications of the screen at once, and a rotated screen are what // same; that, too many duplications of the screen at once, and a rotated screen are what dxgi_open
// dxgi_open returns fall_back for. // returns fall_back for, and Windows.Graphics.Capture (wgc_source.jai) takes over.
// //
// DuplicateOutput1 (IDXGIOutput5) is used where there is one: the plain DuplicateOutput is documented // DuplicateOutput1 (IDXGIOutput5) is used where there is one: the plain DuplicateOutput is documented
// to fail for a process aware of each monitor's DPI, as mcap is. legacy forces the plain one, for // to fail for a process aware of each monitor's DPI, as mcap is (it did not on an RTX 5080 laptop).
// record-probe to check that. // legacy forces the plain one, for record-probe.
Dxgi_Source :: struct { Dxgi_Source :: struct {
adapter: *IDXGIAdapter1; #as using gpu: Capture_Gpu;
output: *IDXGIOutput1; output: *IDXGIOutput1;
output5: *IDXGIOutput5; // null before Windows 10 1703 output5: *IDXGIOutput5; // null before Windows 10 1703
device: *ID3D11Device;
device_context: *ID3D11DeviceContext;
duplication: *IDXGIOutputDuplication; duplication: *IDXGIOutputDuplication;
legacy: bool; legacy: bool;
method: string; // which duplication call worked, for the log method: string; // which duplication call worked, for the log
box: D3D11_BOX; // the region, in the output's pixels pointer_position: POINT; // the cursor's top left, in the screen's pixels
latest: *ID3D11Texture2D; // the region as last seen
staging: [RECORD_STAGING] *ID3D11Texture2D;
copy_ticks: s64; // spent taking images out of frames: QueryInterface and the copy's submission
pointer_position: POINT; // the cursor's top left, in the output's pixels
pointer_visible: bool; pointer_visible: bool;
shape: Cursor_Shape; shape: Cursor_Shape;
} }
RECORD_STAGING :: 3; // staging textures in turn: the one read is two frames behind the one copied
// Every screen attached to the desktop, on every adapter, in DXGI's order. Temp. // Every screen attached to the desktop, on every adapter, in DXGI's order. Temp.
record_list_screens :: () -> [] Record_Screen { record_list_screens :: () -> [] Record_Screen {
screens: [..] Record_Screen; screens: [..] Record_Screen;
@ -58,7 +50,8 @@ record_list_screens :: () -> [] Record_Screen {
IUnknown_Release(output); IUnknown_Release(output);
if desc.AttachedToDesktop != .FALSE { if desc.AttachedToDesktop != .FALSE {
array_add(*screens, .{adapter_index = adapter_index, output_index = output_index, adapter = adapter_name, array_add(*screens, .{adapter_index = adapter_index, output_index = output_index, adapter = adapter_name,
name = wide_text(desc.DeviceName.data), desktop = desc.DesktopCoordinates, rotation = desc.Rotation}); name = wide_text(desc.DeviceName.data), desktop = desc.DesktopCoordinates,
rotation = desc.Rotation, monitor = desc.Monitor});
} }
output_index += 1; output_index += 1;
} }
@ -67,51 +60,22 @@ record_list_screens :: () -> [] Record_Screen {
return screens; return screens;
} }
// Makes the device on the screen's adapter, and duplicates the screen. region is in the screen's // Duplicates the screen. region is in the screen's pixels and lies on it.
// pixels and lies on it.
dxgi_open :: (s: *Dxgi_Source, screen: Record_Screen, region: RECT, legacy: bool) -> ok: bool, error: string, fall_back: bool { dxgi_open :: (s: *Dxgi_Source, screen: Record_Screen, region: RECT, legacy: bool) -> ok: bool, error: string, fall_back: bool {
s.legacy = legacy; s.legacy = legacy;
if screen.rotation != .IDENTITY && screen.rotation != .UNSPECIFIED return false, "the screen is rotated", true; if screen.rotation != .IDENTITY && screen.rotation != .UNSPECIFIED return false, "the screen is rotated", true;
factory: *IDXGIFactory1; ok, error := gpu_open(s, screen, region);
hr := CreateDXGIFactory1(*uid(IDXGIFactory1_UUID), xx *factory); if !ok return false, error, false;
if FAILED(hr) return false, hresult_text("CreateDXGIFactory1", hr), false;
defer IUnknown_Release(factory);
hr = IDXGIFactory1_EnumAdapters1(factory, screen.adapter_index, *s.adapter);
if FAILED(hr) return false, hresult_text("the screen's adapter is gone", hr), false;
output: *IDXGIOutput; output: *IDXGIOutput;
hr = IDXGIAdapter_EnumOutputs(s.adapter, screen.output_index, *output); hr := IDXGIAdapter_EnumOutputs(s.adapter, screen.output_index, *output);
if FAILED(hr) return false, hresult_text("the screen is gone", hr), false; if FAILED(hr) return false, hresult_text("the screen is gone", hr), false;
defer IUnknown_Release(output); defer IUnknown_Release(output);
hr = IUnknown_QueryInterface(output, *uid(IDXGIOutput1_UUID), xx *s.output); hr = IUnknown_QueryInterface(output, *uid(IDXGIOutput1_UUID), xx *s.output);
if FAILED(hr) return false, hresult_text("the screen cannot be duplicated (no IDXGIOutput1)", hr), true; if FAILED(hr) return false, hresult_text("the screen cannot be duplicated (no IDXGIOutput1)", hr), true;
IUnknown_QueryInterface(output, *uid(IDXGIOutput5_UUID), xx *s.output5); // optional IUnknown_QueryInterface(output, *uid(IDXGIOutput5_UUID), xx *s.output5); // optional
levels := D3D_FEATURE_LEVEL.[._11_1, ._11_0, ._10_1, ._10_0];
level: D3D_FEATURE_LEVEL;
hr = D3D11CreateDevice(s.adapter, .UNKNOWN, null, .BGRA_SUPPORT, levels.data, levels.count, D3D11_SDK_VERSION, *s.device, *level, *s.device_context);
if FAILED(hr) return false, hresult_text("D3D11CreateDevice on the screen's adapter", hr), true;
s.box = .{left = xx region.left, top = xx region.top, right = xx region.right, bottom = xx region.bottom, front = 0, back = 1};
width := cast(u32) (region.right - region.left);
height := cast(u32) (region.bottom - region.top);
desc: D3D11_TEXTURE2D_DESC;
desc.Width = width;
desc.Height = height;
desc.MipLevels = 1;
desc.ArraySize = 1;
desc.Format = .B8G8R8A8_UNORM;
desc.SampleDesc.Count = 1;
desc.Usage = .DEFAULT;
hr = ID3D11Device_CreateTexture2D(s.device, *desc, null, *s.latest);
if FAILED(hr) return false, hresult_text("a texture for the region", hr), false;
desc.Usage = .STAGING;
desc.CPUAccessFlags = .READ;
for * s.staging {
hr = ID3D11Device_CreateTexture2D(s.device, *desc, null, it);
if FAILED(hr) return false, hresult_text("a staging texture", hr), false;
}
hr = dxgi_duplicate(s); hr = dxgi_duplicate(s);
if FAILED(hr) { if FAILED(hr) {
// Unsupported: two GPUs, the device on the wrong one. Not currently available: too many duplications. // Unsupported: two GPUs, the device on the wrong one. Not currently available: too many duplications.
@ -134,7 +98,7 @@ dxgi_open :: (s: *Dxgi_Source, screen: Record_Screen, region: RECT, legacy: bool
// Duplicates the screen, again after the duplication was lost. // Duplicates the screen, again after the duplication was lost.
dxgi_duplicate :: (s: *Dxgi_Source) -> HRESULT { dxgi_duplicate :: (s: *Dxgi_Source) -> HRESULT {
if s.duplication { IUnknown_Release(s.duplication); s.duplication = null; } com_release(*s.duplication);
if s.output5 && !s.legacy { if s.output5 && !s.legacy {
s.method = "DuplicateOutput1"; s.method = "DuplicateOutput1";
formats := DXGI_FORMAT.[.B8G8R8A8_UNORM]; formats := DXGI_FORMAT.[.B8G8R8A8_UNORM];
@ -144,11 +108,9 @@ dxgi_duplicate :: (s: *Dxgi_Source) -> HRESULT {
return IDXGIOutput1_DuplicateOutput(s.output, s.device, *s.duplication); return IDXGIOutput1_DuplicateOutput(s.output, s.device, *s.duplication);
} }
Dxgi_Next :: enum u8 { NONE; IMAGE; POINTER; IMAGE_AND_POINTER; LOST; FAILED; } // Waits up to timeout_ms for the screen to change. A new image's region goes into s.latest; a moved
// or changed cursor updates s.pointer_* and s.shape. On LOST the duplication is gone: dxgi_duplicate.
// Waits up to timeout_ms for the screen to change. A new image is copied into s.latest; a moved or dxgi_next :: (s: *Dxgi_Source, timeout_ms: u32) -> Capture_Next, error: string {
// changed cursor updates s.pointer_*. On LOST the duplication is gone: call dxgi_duplicate.
dxgi_next :: (s: *Dxgi_Source, timeout_ms: u32) -> Dxgi_Next, error: string {
info: DXGI_OUTDUPL_FRAME_INFO; info: DXGI_OUTDUPL_FRAME_INFO;
resource: *IDXGIResource; resource: *IDXGIResource;
hr := IDXGIOutputDuplication_AcquireNextFrame(s.duplication, timeout_ms, *info, *resource); hr := IDXGIOutputDuplication_AcquireNextFrame(s.duplication, timeout_ms, *info, *resource);
@ -160,16 +122,11 @@ dxgi_next :: (s: *Dxgi_Source, timeout_ms: u32) -> Dxgi_Next, error: string {
image := info.LastPresentTime.QuadPart != 0; image := info.LastPresentTime.QuadPart != 0;
if image { if image {
start: LARGE_INTEGER;
QueryPerformanceCounter(*start);
texture: *ID3D11Texture2D; texture: *ID3D11Texture2D;
hr = IUnknown_QueryInterface(resource, *uid(ID3D11Texture2D_UUID), xx *texture); hr = IUnknown_QueryInterface(resource, *uid(ID3D11Texture2D_UUID), xx *texture);
if FAILED(hr) return .FAILED, hresult_text("the frame is not a texture", hr); if FAILED(hr) return .FAILED, hresult_text("the frame is not a texture", hr);
ID3D11DeviceContext_CopySubresourceRegion(s.device_context, s.latest, 0, 0, 0, 0, texture, 0, *s.box); gpu_copy_region(s, texture);
IUnknown_Release(texture); IUnknown_Release(texture);
end: LARGE_INTEGER;
QueryPerformanceCounter(*end);
s.copy_ticks += end.QuadPart - start.QuadPart;
} }
pointer := info.LastMouseUpdateTime.QuadPart != 0; pointer := info.LastMouseUpdateTime.QuadPart != 0;
@ -198,41 +155,10 @@ dxgi_next :: (s: *Dxgi_Source, timeout_ms: u32) -> Dxgi_Next, error: string {
} }
dxgi_close :: (s: *Dxgi_Source) { dxgi_close :: (s: *Dxgi_Source) {
release :: (p: *$T) { if p.* { IUnknown_Release(p.*); p.* = null; } } com_release(*s.duplication);
release(*s.duplication); com_release(*s.output5);
for * s.staging release(it); com_release(*s.output);
release(*s.latest); gpu_close(s);
release(*s.device_context);
release(*s.device);
release(*s.output5);
release(*s.output);
release(*s.adapter);
array_free(s.shape.data); array_free(s.shape.data);
s.shape = .{}; s.shape = .{};
} }
#scope_module
hresult_is :: (hr: HRESULT, code: u32) -> bool {
return cast,no_check(u32) hr == code;
}
// "what failed: 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", temp.
hresult_text :: (what: string, hr: HRESULT) -> string {
code := cast,no_check(u32) hr;
name := "";
if code == {
case DXGI_ERROR_UNSUPPORTED; name = " (DXGI_ERROR_UNSUPPORTED)";
case DXGI_ERROR_NOT_CURRENTLY_AVAILABLE; name = " (DXGI_ERROR_NOT_CURRENTLY_AVAILABLE)";
case DXGI_ERROR_ACCESS_LOST; name = " (DXGI_ERROR_ACCESS_LOST)";
case 0x80070005; name = " (E_ACCESSDENIED)";
case 0x80004002; name = " (E_NOINTERFACE)";
case 0x80070057; name = " (E_INVALIDARG)";
}
return tprint("%: 0x%%", what, formatInt(code, base = 16, minimum_digits = 8), name);
}
wide_text :: (wide: *s16) -> string {
text, ok := wide_to_utf8(cast(*u16) wide,, temp);
return ifx ok then text else "";
}

214
modules/Screen_Record/module.jai
View File

@ -2,10 +2,12 @@
// recording and tools/record_probe.jai (video-capture-plan.html). Windows only. // recording and tools/record_probe.jai (video-capture-plan.html). Windows only.
// //
// Two threads. The capture thread takes the screen with DXGI Desktop Duplication (dxgi_source.jai), // Two threads. The capture thread takes the screen with DXGI Desktop Duplication (dxgi_source.jai),
// on a D3D11 device of its own, copies the region on the GPU, reads it back through a ring of staging // or with Windows.Graphics.Capture (wgc_source.jai) where duplication cannot run, each on a D3D11
// textures, so Map never waits on the GPU, and draws the cursor into it (cursor.jai). The encoder // device of its own; copies the region on the GPU; reads it back through a ring of staging textures,
// thread hands the frames to FFmpeg (encoder.jai). Between them is a pool of RECORD_BUFFERS frames; // so Map never waits on the GPU (capture_gpu.jai); and, for duplication, draws the cursor in
// when the encoder falls RECORD_QUEUE frames behind, the oldest waiting is dropped and counted. // (cursor.jai). The encoder thread hands the frames to FFmpeg (encoder.jai). Between them is a pool of
// RECORD_BUFFERS frames; when the encoder falls RECORD_QUEUE frames behind, the oldest waiting is
// dropped and counted.
// //
// A frame is taken at most fps times a second, and only when the screen or the cursor changed, with // A frame is taken at most fps times a second, and only when the screen or the cursor changed, with
// one a second while nothing does, so players can seek. Each carries the time it was taken. // one a second while nothing does, so players can seek. Each carries the time it was taken.
@ -15,9 +17,9 @@
// ... // ...
// ok, error = record_stop(*recorder); // ok, error = record_stop(*recorder);
// //
// fall_back says Desktop Duplication cannot run here: on a laptop with two GPUs, on a rotated screen, // With capture .AUTO (the default), Desktop Duplication is tried first and Windows.Graphics.Capture
// or with too many duplications of the screen running. Windows.Graphics.Capture takes over then // takes over when it cannot run: on a laptop with two GPUs, on a rotated screen, or with too many
// (phase 2 of the plan). // duplications of the screen running. fall_back is only returned for .DXGI, which does not fall back.
RECORD_MIN_WIDTH :: 240; // a smaller region is refused RECORD_MIN_WIDTH :: 240; // a smaller region is refused
RECORD_MIN_HEIGHT :: 180; RECORD_MIN_HEIGHT :: 180;
@ -30,6 +32,20 @@ Record_Screen :: struct {
name: string; // "\\.\DISPLAY1" name: string; // "\\.\DISPLAY1"
desktop: RECT; // on the virtual desktop desktop: RECT; // on the virtual desktop
rotation: DXGI_MODE_ROTATION; rotation: DXGI_MODE_ROTATION;
monitor: HMONITOR;
}
Record_Capture :: enum u8 {
AUTO; // Desktop Duplication, else Windows.Graphics.Capture
DXGI; // Desktop Duplication only
WGC; // Windows.Graphics.Capture only
}
// For record-probe: paths a machine may not take by itself.
Record_Testing :: enum_flags u8 {
DXGI_UNSUPPORTED; // Desktop Duplication fails as on a laptop with two GPUs (DXGI_ERROR_UNSUPPORTED)
WGC_POLL; // Windows.Graphics.Capture polled, as when it will not take the FrameArrived handler
KEEP_BORDER; // Windows.Graphics.Capture's yellow border left on
} }
Record_Options :: struct { Record_Options :: struct {
@ -39,15 +55,18 @@ Record_Options :: struct {
fps: s32 = 30; fps: s32 = 30;
cursor := true; cursor := true;
encoder: string; // "" tries RECORD_ENCODERS in turn encoder: string; // "" tries RECORD_ENCODERS in turn
capture: Record_Capture;
legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe
testing: Record_Testing;
} }
Record_Stats :: struct { Record_Stats :: struct {
source: string; // "DXGI DuplicateOutput1 on <adapter>" source: string; // "DXGI DuplicateOutput1 on <adapter>", "Windows.Graphics.Capture on <adapter>, border off"
fell_back_from: string; // why Desktop Duplication could not run, when it fell back
encoder: string; encoder: string;
width, height: s32; width, height: s32;
captured: s64; // new images of the screen captured: s64; // new images of the screen
pointer_moves: s64; // cursor changes with no new image pointer_moves: s64; // cursor changes with no new image (Desktop Duplication)
emitted: s64; // frames sent to be encoded emitted: s64; // frames sent to be encoded
repeats: s64; // of them, the once-a-second repeat of an unchanged screen repeats: s64; // of them, the once-a-second repeat of an unchanged screen
written: s64; // encoded written: s64; // encoded
@ -60,17 +79,26 @@ Record_Stats :: struct {
Recorder :: struct { Recorder :: struct {
options: Record_Options; options: Record_Options;
source: Dxgi_Source; screen: Record_Screen;
local: RECT; // the region, in the screen's pixels
dxgi: Dxgi_Source; // the capture thread's: one of these is open
wgc: Wgc_Source;
using_wgc: bool;
encoder: Encoder; encoder: Encoder;
width, height: s32; width, height: s32;
frequency: s64; // QueryPerformanceCounter's frequency: s64; // QueryPerformanceCounter's
start_ticks: s64; start_ticks: s64;
capture_thread, encoder_thread: Thread; capture_thread, encoder_thread: Thread;
capture_started, encoder_started: bool;
running: bool; running: bool;
stopping: bool; // set by record_stop; the capture thread ends, then the encoder stopping: bool; // set by record_stop; the capture thread ends, then the encoder
capture_ended: bool; capture_ended: bool;
opened: Semaphore; // the capture thread has opened its source, or failed to
open_ok, open_fall_back: bool;
open_error: string;
mutex: Mutex; mutex: Mutex;
ready: Semaphore; // signalled for each frame queued, and once when capture ends ready: Semaphore; // signalled for each frame queued, and once when capture ends
frames: [RECORD_BUFFERS] Record_Frame; frames: [RECORD_BUFFERS] Record_Frame;
@ -107,48 +135,61 @@ record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: stri
if screen_index < 0 return false, "the region is not on one screen", false; if screen_index < 0 return false, "the region is not on one screen", false;
} }
if screen_index >= screens.count return false, tprint("there is no screen %", screen_index), false; if screen_index >= screens.count return false, tprint("there is no screen %", screen_index), false;
screen := screens[screen_index]; r.screen = screens[screen_index];
if !has_region region = screen.desktop; r.screen.adapter = copy_string(r.screen.adapter);
if !rect_inside(region, screen.desktop) return false, "the region is not on that screen", false; r.screen.name = copy_string(r.screen.name);
if !has_region region = r.screen.desktop;
if !rect_inside(region, r.screen.desktop) return false, "the region is not on that screen", false;
// The minimum first, then even both ways, in the screen's pixels: H.264's 4:2:0 colour needs it. // The minimum first, then even both ways, in the screen's pixels: H.264's 4:2:0 colour needs it.
// RECORD_MIN_WIDTH and _HEIGHT are even, so what passes stays at least that size. // RECORD_MIN_WIDTH and _HEIGHT are even, so what passes stays at least that size.
#assert RECORD_MIN_WIDTH % 2 == 0 && RECORD_MIN_HEIGHT % 2 == 0; #assert RECORD_MIN_WIDTH % 2 == 0 && RECORD_MIN_HEIGHT % 2 == 0;
if region.right - region.left < RECORD_MIN_WIDTH || region.bottom - region.top < RECORD_MIN_HEIGHT if region.right - region.left < RECORD_MIN_WIDTH || region.bottom - region.top < RECORD_MIN_HEIGHT
return false, tprint("% x % is smaller than % x %", region.right - region.left, region.bottom - region.top, RECORD_MIN_WIDTH, RECORD_MIN_HEIGHT), false; return false, tprint("% x % is smaller than % x %", region.right - region.left, region.bottom - region.top, RECORD_MIN_WIDTH, RECORD_MIN_HEIGHT), false;
local := RECT.{region.left - screen.desktop.left, region.top - screen.desktop.top, region.right - screen.desktop.left, region.bottom - screen.desktop.top}; d := r.screen.desktop;
local.right = local.left + ((local.right - local.left) & cast(s32) ~1); r.local = .{region.left - d.left, region.top - d.top, region.right - d.left, region.bottom - d.top};
local.bottom = local.top + ((local.bottom - local.top) & cast(s32) ~1); r.local.right = r.local.left + ((r.local.right - r.local.left) & cast(s32) ~1);
r.width = local.right - local.left; r.local.bottom = r.local.top + ((r.local.bottom - r.local.top) & cast(s32) ~1);
r.height = local.bottom - local.top; r.width = r.local.right - r.local.left;
r.height = r.local.bottom - r.local.top;
ok, error, fall_back := dxgi_open(*r.source, screen, local, options.legacy_duplicate);
if !ok { dxgi_close(*r.source); return false, error, fall_back; }
tried: string;
ok, error, tried = encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder);
if tried log("[record] Encoders that did not open: %", tried);
if !ok { dxgi_close(*r.source); return false, error, false; }
r.stats.source = sprint("DXGI % on %", r.source.method, screen.adapter);
r.stats.encoder = copy_string(r.encoder.name);
r.stats.width, r.stats.height = r.width, r.height; r.stats.width, r.stats.height = r.width, r.height;
init(*r.mutex, "record"); init(*r.mutex, "record");
init(*r.ready); init(*r.ready);
init(*r.opened);
for * r.frames it.pixels = NewArray(cast(s64) r.width * r.height * 4, u8, initialized = false); for * r.frames it.pixels = NewArray(cast(s64) r.width * r.height * 4, u8, initialized = false);
for 0..RECORD_BUFFERS - 1 r.free[it] = cast(s32) it; for 0..RECORD_BUFFERS - 1 r.free[it] = cast(s32) it;
r.free_count = RECORD_BUFFERS; r.free_count = RECORD_BUFFERS;
r.last_frame = NewArray(cast(s64) r.width * r.height * 4, u8); r.last_frame = NewArray(cast(s64) r.width * r.height * 4, u8);
frequency: LARGE_INTEGER; frequency: LARGE_INTEGER;
QueryPerformanceFrequency(*frequency); QueryPerformanceFrequency(*frequency);
r.frequency = frequency.QuadPart; r.frequency = frequency.QuadPart;
r.start_ticks = ticks_now(); r.start_ticks = ticks_now();
// The source is opened on the capture thread, which uses it: WinRT's objects belong to the
// apartment they are made in.
thread_init(*r.capture_thread, record_capture_proc, name = "record capture"); thread_init(*r.capture_thread, record_capture_proc, name = "record capture");
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder"); r.capture_started = true;
r.running = true;
thread_start(*r.capture_thread, r); thread_start(*r.capture_thread, r);
wait_for(*r.opened);
if !r.open_ok {
error, fall_back := copy_string(r.open_error,, temp), r.open_fall_back;
record_free(r);
return false, error, fall_back;
}
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder);
if tried log("[record] Encoders that did not open: %", tried);
if !ok {
error = copy_string(error,, temp);
record_free(r);
return false, error, false;
}
r.stats.encoder = copy_string(r.encoder.name);
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
r.encoder_started = true;
r.running = true;
thread_start(*r.encoder_thread, r); thread_start(*r.encoder_thread, r);
return true, "", false; return true, "", false;
} }
@ -160,19 +201,15 @@ record_stop :: (r: *Recorder) -> ok: bool, error: string {
atomic_swap(*r.stopping, true); atomic_swap(*r.stopping, true);
thread_is_done(*r.capture_thread, -1); thread_is_done(*r.capture_thread, -1);
thread_is_done(*r.encoder_thread, -1); thread_is_done(*r.encoder_thread, -1);
thread_deinit(*r.capture_thread);
thread_deinit(*r.encoder_thread);
r.running = false; r.running = false;
closed, close_error := encoder_close(*r.encoder); closed, close_error := encoder_close(*r.encoder);
r.stats.seconds = cast(float64) (ticks_now() - r.start_ticks) / r.frequency; r.stats.seconds = cast(float64) (ticks_now() - r.start_ticks) / r.frequency;
r.stats.bytes = r.encoder.bytes; r.stats.bytes = r.encoder.bytes;
dxgi_close(*r.source); error := ifx r.error then copy_string(r.error,, temp) else "";
for * r.frames array_free(it.pixels); record_free(r);
destroy(*r.ready);
destroy(*r.mutex);
if r.error return false, r.error; if error return false, error;
if !closed return false, close_error; if !closed return false, close_error;
return true, ""; return true, "";
} }
@ -204,9 +241,64 @@ record_failed :: (r: *Recorder) -> bool, string {
Staged :: struct { pending: bool; ticks: s64; pointer: POINT; pointer_visible: bool; } Staged :: struct { pending: bool; ticks: s64; pointer: POINT; pointer_visible: bool; }
// Stops the capture thread if it runs, and frees what record_start made. The stats' strings are kept.
record_free :: (r: *Recorder) {
atomic_swap(*r.stopping, true);
if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; }
if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; }
if r.encoder.format encoder_close(*r.encoder);
for * r.frames array_free(it.pixels);
array_free(r.last_frame);
r.last_frame = .[];
destroy(*r.opened);
destroy(*r.ready);
destroy(*r.mutex);
free(r.open_error);
free(r.error);
r.open_error, r.error = "", "";
}
// Desktop Duplication, unless asked for Windows.Graphics.Capture, which takes over when duplication
// cannot run here and the options allow.
record_open_source :: (r: *Recorder) -> bool {
if r.options.capture != .WGC {
ok, error, fall_back := dxgi_open(*r.dxgi, r.screen, r.local, r.options.legacy_duplicate);
if ok && r.options.testing & .DXGI_UNSUPPORTED {
ok, error, fall_back = false, "Desktop Duplication (testing): 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", true;
}
if ok {
r.stats.source = sprint("DXGI % on %", r.dxgi.method, r.screen.adapter);
return true;
}
dxgi_close(*r.dxgi);
if r.options.capture == .DXGI || !fall_back {
r.open_error, r.open_fall_back = copy_string(error), fall_back;
return false;
}
log("[record] Desktop Duplication cannot run here (%); Windows.Graphics.Capture instead.", error);
r.stats.fell_back_from = copy_string(error);
}
ok, error := wgc_open(*r.wgc, r.screen, r.local, r.options.cursor, r.options.testing);
if !ok {
wgc_close(*r.wgc);
r.open_error = copy_string(error);
return false;
}
r.using_wgc = true;
r.stats.source = sprint("Windows.Graphics.Capture on %, %%", r.screen.adapter, r.wgc.border, ifx r.wgc.polling then ", polled" else "");
return true;
}
record_capture_proc :: (thread: *Thread) -> s64 { record_capture_proc :: (thread: *Thread) -> s64 {
r := cast(*Recorder) thread.data; r := cast(*Recorder) thread.data;
s := *r.source; wgc_thread_start();
defer wgc_thread_end();
r.open_ok = record_open_source(r);
signal(*r.opened);
if !r.open_ok return 0;
gpu: *Capture_Gpu = ifx r.using_wgc then *r.wgc.gpu else *r.dxgi.gpu;
defer if r.using_wgc wgc_close(*r.wgc); else dxgi_close(*r.dxgi);
// A change is sent as soon as it comes once its turn is due: turns are a frame's time apart on a // A change is sent as soon as it comes once its turn is due: turns are a frame's time apart on a
// schedule that moves on a frame at a time, so the rate never passes fps, and a tenth of a frame // schedule that moves on a frame at a time, so the rate never passes fps, and a tenth of a frame
// early counts as on time, so a 60 Hz screen's frames, which jitter, all make a 60 fps cap. // early counts as on time, so a 60 Hz screen's frames, which jitter, all make a 60 fps cap.
@ -221,8 +313,11 @@ record_capture_proc :: (thread: *Thread) -> s64 {
while !atomic_read(*r.stopping) { while !atomic_read(*r.stopping) {
now := ticks_now(); now := ticks_now();
due := ifx changed then turn - early else last_emit + r.frequency; // the change, or the repeat due := ifx changed then turn - early else last_emit + r.frequency; // the change, or the repeat
wait := clamp((due - now) * 1000 / r.frequency, 0, 100); wait := cast(u32) clamp((due - now) * 1000 / r.frequency, 0, 100);
next, error := dxgi_next(s, cast(u32) wait); next: Capture_Next;
error: string;
if r.using_wgc next, error = wgc_next(*r.wgc, wait);
else next, error = dxgi_next(*r.dxgi, wait);
if next == { if next == {
case .FAILED; case .FAILED;
record_fail(r, error); record_fail(r, error);
@ -231,7 +326,7 @@ record_capture_proc :: (thread: *Thread) -> s64 {
// A mode change, the secure desktop, a game going fullscreen: duplicated again, and on. // A mode change, the secure desktop, a game going fullscreen: duplicated again, and on.
lock(*r.mutex); r.stats.lost += 1; unlock(*r.mutex); lock(*r.mutex); r.stats.lost += 1; unlock(*r.mutex);
while !atomic_read(*r.stopping) { while !atomic_read(*r.stopping) {
hr := dxgi_duplicate(s); hr := dxgi_duplicate(*r.dxgi);
if SUCCEEDED(hr) break; if SUCCEEDED(hr) break;
if !hresult_is(hr, DXGI_ERROR_ACCESS_LOST) && !hresult_is(hr, 0x80070005) && !hresult_is(hr, DXGI_ERROR_NOT_CURRENTLY_AVAILABLE) { if !hresult_is(hr, DXGI_ERROR_ACCESS_LOST) && !hresult_is(hr, 0x80070005) && !hresult_is(hr, DXGI_ERROR_NOT_CURRENTLY_AVAILABLE) {
record_fail(r, hresult_text("the screen could not be duplicated again", hr)); record_fail(r, hresult_text("the screen could not be duplicated again", hr));
@ -246,7 +341,7 @@ record_capture_proc :: (thread: *Thread) -> s64 {
have_image, changed = true, true; have_image, changed = true, true;
lock(*r.mutex); lock(*r.mutex);
r.stats.captured += 1; r.stats.captured += 1;
r.stats.copy_ticks = s.copy_ticks; r.stats.copy_ticks = gpu.copy_ticks;
unlock(*r.mutex); unlock(*r.mutex);
case .POINTER; case .POINTER;
if r.options.cursor && have_image { if r.options.cursor && have_image {
@ -263,9 +358,10 @@ record_capture_proc :: (thread: *Thread) -> s64 {
// Copied to staging now; read back two frames on, when the GPU is done with it. // Copied to staging now; read back two frames on, when the GPU is done with it.
slot := emits % RECORD_STAGING; slot := emits % RECORD_STAGING;
if staged[slot].pending record_read_back(r, slot, staged[slot]); if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
ID3D11DeviceContext_CopyResource(s.device_context, s.staging[slot], s.latest); ID3D11DeviceContext_CopyResource(gpu.device_context, gpu.staging[slot], gpu.latest);
staged[slot] = .{pending = true, ticks = now, pointer = s.pointer_position, pointer_visible = s.pointer_visible}; staged[slot] = .{pending = true, ticks = now};
if !r.using_wgc staged[slot].pointer, staged[slot].pointer_visible = r.dxgi.pointer_position, r.dxgi.pointer_visible;
emits += 1; emits += 1;
last_emit = now; last_emit = now;
changed = false; changed = false;
@ -278,16 +374,16 @@ record_capture_proc :: (thread: *Thread) -> s64 {
// What is still staged, oldest first. // What is still staged, oldest first.
for 0..RECORD_STAGING - 1 { for 0..RECORD_STAGING - 1 {
slot := (emits + it) % RECORD_STAGING; slot := (emits + it) % RECORD_STAGING;
if staged[slot].pending record_read_back(r, slot, staged[slot]); if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
} }
atomic_swap(*r.capture_ended, true); atomic_swap(*r.capture_ended, true);
signal(*r.ready); signal(*r.ready);
return 0; return 0;
} }
// Copies a staging texture into a free frame, draws the cursor and queues it. // Copies a staging texture into a free frame, draws the cursor (Desktop Duplication leaves it out)
record_read_back :: (r: *Recorder, slot: s64, staged: Staged) { // and queues it.
s := *r.source; record_read_back :: (r: *Recorder, gpu: *Capture_Gpu, slot: s64, staged: Staged) {
t := ticks_now(); t := ticks_now();
lock(*r.mutex); lock(*r.mutex);
if !r.free_count { // every frame is waiting: drop the oldest if !r.free_count { // every frame is waiting: drop the oldest
@ -303,7 +399,7 @@ record_read_back :: (r: *Recorder, slot: s64, staged: Staged) {
frame := *r.frames[index]; frame := *r.frames[index];
mapped: D3D11_MAPPED_SUBRESOURCE; mapped: D3D11_MAPPED_SUBRESOURCE;
hr := ID3D11DeviceContext_Map(s.device_context, s.staging[slot], 0, .READ, 0, *mapped); hr := ID3D11DeviceContext_Map(gpu.device_context, gpu.staging[slot], 0, .READ, 0, *mapped);
if FAILED(hr) { if FAILED(hr) {
lock(*r.mutex); r.free[r.free_count] = index; r.free_count += 1; unlock(*r.mutex); lock(*r.mutex); r.free[r.free_count] = index; r.free_count += 1; unlock(*r.mutex);
record_fail(r, hresult_text("reading the frame back", hr)); record_fail(r, hresult_text("reading the frame back", hr));
@ -311,10 +407,10 @@ record_read_back :: (r: *Recorder, slot: s64, staged: Staged) {
} }
row := cast(s64) r.width * 4; row := cast(s64) r.width * 4;
for y: 0..r.height - 1 memcpy(frame.pixels.data + y * row, cast(*u8) mapped.pData + cast(s64) y * mapped.RowPitch, row); for y: 0..r.height - 1 memcpy(frame.pixels.data + y * row, cast(*u8) mapped.pData + cast(s64) y * mapped.RowPitch, row);
ID3D11DeviceContext_Unmap(s.device_context, s.staging[slot], 0); ID3D11DeviceContext_Unmap(gpu.device_context, gpu.staging[slot], 0);
if r.options.cursor && staged.pointer_visible { if r.options.cursor && staged.pointer_visible {
cursor_draw(frame.pixels.data, r.width, r.height, *s.shape, cursor_draw(frame.pixels.data, r.width, r.height, *r.dxgi.shape,
staged.pointer.x - cast(s32) s.box.left, staged.pointer.y - cast(s32) s.box.top); staged.pointer.x - cast(s32) gpu.box.left, staged.pointer.y - cast(s32) gpu.box.top);
} }
frame.ticks = staged.ticks; frame.ticks = staged.ticks;
@ -383,19 +479,15 @@ record_fail :: (r: *Recorder, error: string) {
atomic_swap(*r.stopping, true); atomic_swap(*r.stopping, true);
} }
ticks_now :: () -> s64 {
t: LARGE_INTEGER;
QueryPerformanceCounter(*t);
return t.QuadPart;
}
rect_inside :: (inner: RECT, outer: RECT) -> bool { rect_inside :: (inner: RECT, outer: RECT) -> bool {
return inner.left >= outer.left && inner.top >= outer.top && inner.right <= outer.right && inner.bottom <= outer.bottom; return inner.left >= outer.left && inner.top >= outer.top && inner.right <= outer.right && inner.bottom <= outer.bottom;
} }
#scope_export #scope_export
#load "capture_gpu.jai";
#load "dxgi_source.jai"; #load "dxgi_source.jai";
#load "wgc_source.jai";
#load "encoder.jai"; #load "encoder.jai";
#load "cursor.jai"; #load "cursor.jai";

406
modules/Screen_Record/wgc_source.jai Normal file
View File

@ -0,0 +1,406 @@
// Windows.Graphics.Capture: the screen as Windows' own capture API hands it over, for when Desktop
// Duplication cannot run (dxgi_source.jai). It works across GPUs, hands over frames the right way up
// on a rotated screen, and draws the cursor in itself.
//
// It is WinRT, bound here by hand: the few interfaces it needs, as vtables after IInspectable's six
// methods. Everything is made and used on the capture thread, which is in the multithreaded apartment
// (RoInitialize), with a free-threaded frame pool, so no dispatcher queue or message loop is needed.
//
// A frame's arrival sets an event, through a small handler object (Frame_Handler); if Windows will not
// take the handler, the pool is polled every millisecond instead. Windows 11 lets the yellow border it
// draws around a screen being captured be turned off (IsBorderRequired); Windows 10 always draws it.
Wgc_Source :: struct {
#as using gpu: Capture_Gpu;
screen_size: Size_Int32; // the frames' size, as the pool was made for
item: *Capture_Item;
winrt_device: *IInspectable; // IDirect3DDevice, over gpu.device
pool: *Frame_Pool;
session: *Capture_Session;
handler: Frame_Handler;
handler_token: s64;
polling: bool; // Windows would not take the handler
border: string; // what came of turning the border off, for the log
}
// Starts capturing the screen. region is in the screen's pixels and lies on it.
wgc_open :: (s: *Wgc_Source, screen: Record_Screen, region: RECT, cursor: bool, testing: Record_Testing = 0) -> ok: bool, error: string {
ok, error := gpu_open(s, screen, region);
if !ok return false, error;
session_statics: *Session_Statics;
hr := activation_factory("Windows.Graphics.Capture.GraphicsCaptureSession", IID_SESSION_STATICS, xx *session_statics);
if FAILED(hr) return false, hresult_text("Windows.Graphics.Capture is not on this system", hr);
supported: u8;
session_statics.vtable.IsSupported(session_statics, *supported);
com_release(*session_statics);
if !supported return false, "Windows.Graphics.Capture is not supported here";
interop: *Capture_Item_Interop;
hr = activation_factory("Windows.Graphics.Capture.GraphicsCaptureItem", IID_ITEM_INTEROP, xx *interop);
if FAILED(hr) return false, hresult_text("IGraphicsCaptureItemInterop", hr);
hr = interop.vtable.CreateForMonitor(interop, screen.monitor, *IID_ITEM, xx *s.item);
com_release(*interop);
if FAILED(hr) return false, hresult_text("a capture item for the screen", hr);
s.item.vtable.get_Size(s.item, *s.screen_size);
desktop_width := screen.desktop.right - screen.desktop.left;
desktop_height := screen.desktop.bottom - screen.desktop.top;
if s.screen_size.width != desktop_width || s.screen_size.height != desktop_height {
return false, tprint("the screen is % x %, but this process sees it as % x %: it must be aware of each monitor's DPI",
s.screen_size.width, s.screen_size.height, desktop_width, desktop_height);
}
dxgi_device: *IDXGIDevice;
hr = IUnknown_QueryInterface(s.device, *uid(IDXGIDevice_UUID), xx *dxgi_device);
if FAILED(hr) return false, hresult_text("the device's IDXGIDevice", hr);
inspectable: *IInspectable;
hr = CreateDirect3D11DeviceFromDXGIDevice(dxgi_device, *inspectable);
com_release(*dxgi_device);
if FAILED(hr) return false, hresult_text("CreateDirect3D11DeviceFromDXGIDevice", hr);
hr = inspectable.vtable.QueryInterface(inspectable, *IID_DIRECT3D_DEVICE, xx *s.winrt_device);
com_release(*inspectable);
if FAILED(hr) return false, hresult_text("the device as IDirect3DDevice", hr);
pool_statics: *Frame_Pool_Statics2;
hr = activation_factory("Windows.Graphics.Capture.Direct3D11CaptureFramePool", IID_POOL_STATICS2, xx *pool_statics);
if FAILED(hr) return false, hresult_text("Direct3D11CaptureFramePool's statics", hr);
hr = pool_statics.vtable.CreateFreeThreaded(pool_statics, s.winrt_device, PIXEL_FORMAT_B8G8R8A8, POOL_BUFFERS, size_packed(s.screen_size), *s.pool);
com_release(*pool_statics);
if FAILED(hr) return false, hresult_text("a free-threaded frame pool", hr);
hr = s.pool.vtable.CreateCaptureSession(s.pool, s.item, *s.session);
if FAILED(hr) return false, hresult_text("a capture session", hr);
// The cursor (Windows 10 2004 on) and the border (Windows 11): optional, by what the system has.
session2: *Capture_Session2;
if SUCCEEDED(s.session.vtable.QueryInterface(s.session, *IID_SESSION2, xx *session2)) {
session2.vtable.put_IsCursorCaptureEnabled(session2, xx cursor);
com_release(*session2);
}
session3: *Capture_Session3;
if testing & .KEEP_BORDER {
s.border = "border left on";
} else if SUCCEEDED(s.session.vtable.QueryInterface(s.session, *IID_SESSION3, xx *session3)) {
hr = session3.vtable.put_IsBorderRequired(session3, 0);
s.border = ifx SUCCEEDED(hr) then "border off" else hresult_text("the border could not be turned off", hr);
com_release(*session3);
} else {
s.border = "border always drawn (no IGraphicsCaptureSession3: Windows 10)";
}
s.handler.vtable = *frame_handler_vtable;
s.handler.event = CreateEventW(null, 0, 0, null);
hr = ifx testing & .WGC_POLL then cast,no_check(HRESULT) 0x80004002 else s.pool.vtable.add_FrameArrived(s.pool, *s.handler, *s.handler_token);
if FAILED(hr) {
s.polling = true;
timeBeginPeriod(1); // so polling every millisecond is every millisecond
log("[record] FrameArrived would not take the handler (%); polling the frame pool.", hresult_text("add_FrameArrived", hr));
}
hr = s.session.vtable.StartCapture(s.session);
if FAILED(hr) return false, hresult_text("StartCapture", hr);
return true, "";
}
// Waits up to timeout_ms for a new frame; the newest's region goes into s.latest.
wgc_next :: (s: *Wgc_Source, timeout_ms: u32) -> Capture_Next, error: string {
frame := wgc_take_newest(s);
if !frame {
if s.polling {
start := ticks_now();
frequency: LARGE_INTEGER;
QueryPerformanceFrequency(*frequency);
while !frame && (ticks_now() - start) * 1000 / frequency.QuadPart < timeout_ms {
sleep_milliseconds(1);
frame = wgc_take_newest(s);
}
} else {
WaitForSingleObject(s.handler.event, timeout_ms);
frame = wgc_take_newest(s);
}
}
if !frame return .NONE, "";
defer close_and_release(frame);
// The screen changed size (a new resolution): the pool is made again; the region must still fit.
size: Size_Int32;
frame.vtable.get_ContentSize(frame, *size);
if size.width != s.screen_size.width || size.height != s.screen_size.height {
s.screen_size = size;
hr := s.pool.vtable.Recreate(s.pool, s.winrt_device, PIXEL_FORMAT_B8G8R8A8, POOL_BUFFERS, size_packed(size));
if FAILED(hr) return .FAILED, hresult_text("the frame pool could not be made again", hr);
if cast(s32) s.box.right > size.width || cast(s32) s.box.bottom > size.height
return .FAILED, tprint("the screen is now % x %, and the region is not on it", size.width, size.height);
return .NONE, "";
}
surface: *IInspectable;
hr := frame.vtable.get_Surface(frame, *surface);
if FAILED(hr) return .FAILED, hresult_text("the frame's surface", hr);
defer com_release(*surface);
access: *Dxgi_Interface_Access;
hr = surface.vtable.QueryInterface(surface, *IID_DXGI_INTERFACE_ACCESS, xx *access);
if FAILED(hr) return .FAILED, hresult_text("the surface's IDirect3DDxgiInterfaceAccess", hr);
defer com_release(*access);
texture: *ID3D11Texture2D;
hr = access.vtable.GetInterface(access, *uid(ID3D11Texture2D_UUID), xx *texture);
if FAILED(hr) return .FAILED, hresult_text("the surface's texture", hr);
gpu_copy_region(s, texture);
IUnknown_Release(texture);
return .IMAGE, "";
}
wgc_close :: (s: *Wgc_Source) {
if s.pool && s.handler_token s.pool.vtable.remove_FrameArrived(s.pool, s.handler_token);
if s.session close_and_release(s.session);
if s.pool close_and_release(s.pool);
s.session, s.pool = null, null;
com_release(*s.item);
com_release(*s.winrt_device);
if s.handler.event CloseHandle(s.handler.event);
if s.polling timeEndPeriod(1);
gpu_close(s);
s.* = .{};
}
// The thread's apartment: WinRT on this thread from here on.
wgc_thread_start :: () -> ok: bool {
if !winrt_load() return false;
return SUCCEEDED(winrt.RoInitialize(RO_INIT_MULTITHREADED));
}
wgc_thread_end :: () {
if winrt.RoUninitialize winrt.RoUninitialize();
}
#scope_file
POOL_BUFFERS :: 2;
PIXEL_FORMAT_B8G8R8A8 :: 87; // DirectXPixelFormat.B8G8R8A8UIntNormalized
RO_INIT_MULTITHREADED :: 1;
// The pool's newest frame, the older ones let go; null when there is none.
wgc_take_newest :: (s: *Wgc_Source) -> *Capture_Frame {
newest: *Capture_Frame;
while true {
frame: *Capture_Frame;
if FAILED(s.pool.vtable.TryGetNextFrame(s.pool, *frame)) || !frame break;
if newest close_and_release(newest);
newest = frame;
}
return newest;
}
// IClosable.Close, which hands a frame back to the pool at once, then Release.
close_and_release :: (object: *$T) {
closable: *Closable;
if SUCCEEDED(object.vtable.QueryInterface(object, *IID_CLOSABLE, xx *closable)) {
closable.vtable.Close(closable);
closable.vtable.Release(closable);
}
object.vtable.Release(object);
}
activation_factory :: (class_name: string, iid: GUID, factory: **void) -> HRESULT {
wide := utf8_to_wide(class_name,, temp);
length: u32;
while wide[length] length += 1;
name: HSTRING;
if !winrt.loaded return cast,no_check(HRESULT) 0x80004001; // E_NOTIMPL: no WinRT on this system
hr := winrt.WindowsCreateString(wide, length, *name);
if FAILED(hr) return hr;
defer winrt.WindowsDeleteString(name);
iid_copy := iid;
return winrt.RoGetActivationFactory(name, *iid_copy, factory);
}
size_packed :: (size: Size_Int32) -> u64 { // SizeInt32 by value: 8 bytes, in one register
width := cast(u64) cast,no_check(u32) size.width;
height := cast(u64) cast,no_check(u32) size.height;
return width | (height << 32);
}
// ---------------------------------------------------------------- the FrameArrived handler
// TypedEventHandler<Direct3D11CaptureFramePool, Object>: a COM object of our own whose Invoke sets an
// event. It lives in the Wgc_Source and is never freed by its count; it is unregistered first.
Frame_Handler :: struct {
vtable: *Frame_Handler_VTable;
event: HANDLE;
}
Frame_Handler_VTable :: struct {
QueryInterface: (this: *Frame_Handler, iid: *GUID, out: **void) -> HRESULT #c_call;
AddRef: (this: *Frame_Handler) -> u32 #c_call;
Release: (this: *Frame_Handler) -> u32 #c_call;
Invoke: (this: *Frame_Handler, sender: *void, args: *void) -> HRESULT #c_call;
}
frame_handler_vtable := Frame_Handler_VTable.{handler_query, handler_add_ref, handler_release, handler_invoke};
handler_query :: (this: *Frame_Handler, iid: *GUID, out: **void) -> HRESULT #c_call {
if guid_equal(iid, *IID_UNKNOWN) || guid_equal(iid, *IID_AGILE_OBJECT) || guid_equal(iid, *IID_FRAME_ARRIVED_HANDLER) {
out.* = this;
return 0;
}
out.* = null;
return cast,no_check(HRESULT) 0x80004002; // E_NOINTERFACE
}
handler_add_ref :: (this: *Frame_Handler) -> u32 #c_call { return 1; }
handler_release :: (this: *Frame_Handler) -> u32 #c_call { return 1; }
handler_invoke :: (this: *Frame_Handler, sender: *void, args: *void) -> HRESULT #c_call {
SetEvent(this.event);
return 0;
}
guid_equal :: (a: *GUID, b: *GUID) -> bool #c_call {
if a.Data1 != b.Data1 || a.Data2 != b.Data2 || a.Data3 != b.Data3 return false;
for 0..7 if a.Data4[it] != b.Data4[it] return false;
return true;
}
// ---------------------------------------------------------------- bindings
d3d11_dll :: #library,system "d3d11";
kernel32_dll :: #library,system "kernel32";
HSTRING :: *void;
// WinRT's own calls, from combase.dll at run time: its import library is runtimeobject.lib, which
// names no DLL of its own for #library to find.
winrt: struct {
loaded: bool;
RoInitialize: (init_type: s32) -> HRESULT #c_call;
RoUninitialize: () #c_call;
RoGetActivationFactory: (class_id: HSTRING, iid: *GUID, factory: **void) -> HRESULT #c_call;
WindowsCreateString: (source: *u16, length: u32, out: *HSTRING) -> HRESULT #c_call;
WindowsDeleteString: (text: HSTRING) -> HRESULT #c_call;
}
winrt_load :: () -> bool {
if winrt.loaded return true;
library := LoadLibraryW(utf8_to_wide("combase.dll",, temp));
if !library return false;
winrt.RoInitialize = xx GetProcAddress(library, "RoInitialize");
winrt.RoUninitialize = xx GetProcAddress(library, "RoUninitialize");
winrt.RoGetActivationFactory = xx GetProcAddress(library, "RoGetActivationFactory");
winrt.WindowsCreateString = xx GetProcAddress(library, "WindowsCreateString");
winrt.WindowsDeleteString = xx GetProcAddress(library, "WindowsDeleteString");
winrt.loaded = winrt.RoInitialize && winrt.RoUninitialize && winrt.RoGetActivationFactory && winrt.WindowsCreateString && winrt.WindowsDeleteString;
return winrt.loaded;
}
CreateDirect3D11DeviceFromDXGIDevice :: (device: *IDXGIDevice, out: **IInspectable) -> HRESULT #foreign d3d11_dll;
SetEvent :: (event: HANDLE) -> s32 #foreign kernel32_dll;
Size_Int32 :: struct { width, height: s32; }
IID_UNKNOWN := #run guid("00000000-0000-0000-C000-000000000046");
IID_AGILE_OBJECT := #run guid("94ea2b94-e9cc-49e0-c0ff-ee64ca8f5b90");
IID_FRAME_ARRIVED_HANDLER := #run guid("51a947f7-79cf-5a3e-a3a5-1289cfa6dfe8"); // TypedEventHandler<Direct3D11CaptureFramePool, Object>
IID_ITEM_INTEROP := #run guid("3628E81B-3CAC-4C60-B7F4-23CE0E0C3356");
IID_ITEM := #run guid("79C3F95B-31F7-4EC2-A464-632EF5D30760");
IID_SESSION_STATICS := #run guid("2224A540-5974-49AA-B232-0882536F4CB5");
IID_SESSION2 := #run guid("2C39AE40-7D2E-5044-804E-8B6799D4CF9E");
IID_SESSION3 := #run guid("F2CDD966-22AE-5EA1-9596-3A289344C3BE");
IID_POOL_STATICS2 := #run guid("589B103F-6BBC-5DF5-A991-02E28B3B66D5");
IID_DIRECT3D_DEVICE := #run guid("A37624AB-8D5F-4650-9D3E-9EAE3D9BC670");
IID_DXGI_INTERFACE_ACCESS := #run guid("A9B3D012-3DF2-4EE3-B8D1-8695F457D3C1");
IID_CLOSABLE := #run guid("30D5A829-7FA4-4026-83BB-D75BAE4EA99E");
guid :: (text: string) -> GUID {
hex :: (s: string) -> u64 {
value: u64;
for 0..s.count - 1 {
c := s[it];
digit: u64 = ifx c >= #char "a" then cast(u64) (c - #char "a" + 10) else ifx c >= #char "A" then cast(u64) (c - #char "A" + 10) else cast(u64) (c - #char "0");
value = value * 16 + digit;
}
return value;
}
g: GUID;
g.Data1 = cast(u32) hex(slice(text, 0, 8));
g.Data2 = cast(u16) hex(slice(text, 9, 4));
g.Data3 = cast(u16) hex(slice(text, 14, 4));
for 0..1 g.Data4[it] = cast(u8) hex(slice(text, 19 + it * 2, 2));
for 0..5 g.Data4[it + 2] = cast(u8) hex(slice(text, 24 + it * 2, 2));
return g;
}
// IUnknown, then IInspectable's three: every WinRT interface starts so.
Inspectable_Methods :: struct {
QueryInterface: (this: *void, iid: *GUID, out: **void) -> HRESULT #c_call;
AddRef: (this: *void) -> u32 #c_call;
Release: (this: *void) -> u32 #c_call;
GetIids: *void;
GetRuntimeClassName: *void;
GetTrustLevel: *void;
}
IInspectable :: struct { vtable: *Inspectable_Methods; }
Closable :: struct { vtable: *struct { using base: Inspectable_Methods; Close: (this: *Closable) -> HRESULT #c_call; }; }
Session_Statics :: struct { vtable: *struct { using base: Inspectable_Methods; IsSupported: (this: *Session_Statics, out: *u8) -> HRESULT #c_call; }; }
Capture_Item_Interop :: struct { vtable: *struct {
QueryInterface: (this: *void, iid: *GUID, out: **void) -> HRESULT #c_call;
AddRef: (this: *void) -> u32 #c_call;
Release: (this: *void) -> u32 #c_call;
CreateForWindow: (this: *Capture_Item_Interop, window: HWND, iid: *GUID, out: **void) -> HRESULT #c_call;
CreateForMonitor: (this: *Capture_Item_Interop, monitor: HMONITOR, iid: *GUID, out: **void) -> HRESULT #c_call;
}; }
Capture_Item :: struct { vtable: *struct {
using base: Inspectable_Methods;
get_DisplayName: *void;
get_Size: (this: *Capture_Item, out: *Size_Int32) -> HRESULT #c_call;
add_Closed: *void;
remove_Closed: *void;
}; }
Frame_Pool_Statics2 :: struct { vtable: *struct {
using base: Inspectable_Methods;
CreateFreeThreaded: (this: *Frame_Pool_Statics2, device: *IInspectable, format: s32, buffers: s32, size: u64, out: **Frame_Pool) -> HRESULT #c_call;
}; }
Frame_Pool :: struct { vtable: *struct {
using base: Inspectable_Methods;
Recreate: (this: *Frame_Pool, device: *IInspectable, format: s32, buffers: s32, size: u64) -> HRESULT #c_call;
TryGetNextFrame: (this: *Frame_Pool, out: **Capture_Frame) -> HRESULT #c_call;
add_FrameArrived: (this: *Frame_Pool, handler: *Frame_Handler, token: *s64) -> HRESULT #c_call;
remove_FrameArrived: (this: *Frame_Pool, token: s64) -> HRESULT #c_call;
CreateCaptureSession: (this: *Frame_Pool, item: *Capture_Item, out: **Capture_Session) -> HRESULT #c_call;
get_DispatcherQueue: *void;
}; }
Capture_Frame :: struct { vtable: *struct {
using base: Inspectable_Methods;
get_Surface: (this: *Capture_Frame, out: **IInspectable) -> HRESULT #c_call;
get_SystemRelativeTime: (this: *Capture_Frame, out: *s64) -> HRESULT #c_call;
get_ContentSize: (this: *Capture_Frame, out: *Size_Int32) -> HRESULT #c_call;
}; }
Capture_Session :: struct { vtable: *struct {
using base: Inspectable_Methods;
StartCapture: (this: *Capture_Session) -> HRESULT #c_call;
}; }
Capture_Session2 :: struct { vtable: *struct {
using base: Inspectable_Methods;
get_IsCursorCaptureEnabled: (this: *Capture_Session2, out: *u8) -> HRESULT #c_call;
put_IsCursorCaptureEnabled: (this: *Capture_Session2, value: u8) -> HRESULT #c_call;
}; }
Capture_Session3 :: struct { vtable: *struct {
using base: Inspectable_Methods;
get_IsBorderRequired: (this: *Capture_Session3, out: *u8) -> HRESULT #c_call;
put_IsBorderRequired: (this: *Capture_Session3, value: u8) -> HRESULT #c_call;
}; }
Dxgi_Interface_Access :: struct { vtable: *struct {
QueryInterface: (this: *void, iid: *GUID, out: **void) -> HRESULT #c_call;
AddRef: (this: *void) -> u32 #c_call;
Release: (this: *void) -> u32 #c_call;
GetInterface: (this: *Dxgi_Interface_Access, iid: *GUID, out: **void) -> HRESULT #c_call;
}; }

104
tools/record_probe.jai
View File

@ -3,6 +3,14 @@
// there are and on which adapter, which duplication call and encoder worked, frames captured, written // there are and on which adapter, which duplication call and encoder worked, frames captured, written
// and dropped, time per stage, and the file's size. It also answers what the plan leaves to phase 1: // and dropped, time per stage, and the file's size. It also answers what the plan leaves to phase 1:
// //
// -capture HOW auto (Desktop Duplication, else Windows.Graphics.Capture), dxgi or wgc.
// -border-test counts yellow pixels along the screen's edges, with GDI, before recording and a
// second in: does Windows draw its capture border (Windows.Graphics.Capture)?
// -keep-border leaves Windows.Graphics.Capture's border on: the border test's control.
// -simulate-unsupported Desktop Duplication fails as on a laptop with two GPUs: the fallback's path.
// -poll Windows.Graphics.Capture polled, as when it will not take its frame handler.
// -cursor-test puts the mouse in the middle of the region while recording, and back after;
// with -snapshot, cursor.png is the frame around it, four times as large.
// -legacy the plain DuplicateOutput, not DuplicateOutput1: does it fail for a process aware // -legacy the plain DuplicateOutput, not DuplicateOutput1: does it fail for a process aware
// of each monitor's DPI, as mcap is (this probe is, unless -dpi-unaware)? // of each monitor's DPI, as mcap is (this probe is, unless -dpi-unaware)?
// -cuda makes a CUDA context first, as onnxruntime does in mcap: does duplication still work? // -cuda makes a CUDA context first, as onnxruntime does in mcap: does duplication still work?
@ -18,7 +26,8 @@
// Build: jai first.jai - record_probe // Build: jai first.jai - record_probe
// Run: bin/record-probe [-list] [-screen N] [-region X Y W H] [-seconds S] [-fps F] [-encoder NAME] // Run: bin/record-probe [-list] [-screen N] [-region X Y W H] [-seconds S] [-fps F] [-encoder NAME]
// [-out FILE.mp4] [-no-cursor] [-legacy] [-cuda] [-exclude-test] [-snapshot DIR] // [-out FILE.mp4] [-no-cursor] [-legacy] [-cuda] [-exclude-test] [-snapshot DIR]
// [-animate] [-dpi-unaware] // [-animate] [-capture auto|dxgi|wgc] [-border-test] [-keep-border] [-cursor-test]
// [-simulate-unsupported] [-poll] [-dpi-unaware]
// Exit code: 0 recorded, 1 failed, 3 Desktop Duplication cannot run here (the fallback's case). // Exit code: 0 recorded, 1 failed, 3 Desktop Duplication cannot run here (the fallback's case).
main :: () { main :: () {
@ -26,7 +35,7 @@ main :: () {
options: Record_Options; options: Record_Options;
options.path = tprint("%/record-probe.mp4", get_working_directory()); options.path = tprint("%/record-probe.mp4", get_working_directory());
seconds: float64 = 10; seconds: float64 = 10;
list_only, cuda, exclude_test, dpi_unaware, animate := false, false, false, false, false; list_only, cuda, exclude_test, dpi_unaware, animate, border_test, cursor_test := false, false, false, false, false, false, false;
snapshot_folder: string; snapshot_folder: string;
i := 1; i := 1;
number :: (index: s64) -> s32 #expand { return cast(s32) string_to_int(`args[index]); } number :: (index: s64) -> s32 #expand { return cast(s32) string_to_int(`args[index]); }
@ -51,6 +60,19 @@ main :: () {
if a == "-exclude-test" { exclude_test = true; i += 1; continue; } if a == "-exclude-test" { exclude_test = true; i += 1; continue; }
if a == "-dpi-unaware" { dpi_unaware = true; i += 1; continue; } if a == "-dpi-unaware" { dpi_unaware = true; i += 1; continue; }
if a == "-animate" { animate = true; i += 1; continue; } if a == "-animate" { animate = true; i += 1; continue; }
if a == "-border-test" { border_test = true; i += 1; continue; }
if a == "-keep-border" { options.testing |= .KEEP_BORDER; i += 1; continue; }
if a == "-simulate-unsupported" { options.testing |= .DXGI_UNSUPPORTED; i += 1; continue; }
if a == "-poll" { options.testing |= .WGC_POLL; i += 1; continue; }
if a == "-cursor-test" { cursor_test = true; i += 1; continue; }
if a == "-capture" && more >= 1 {
how := args[i + 1];
if how == "auto" options.capture = .AUTO;
else if how == "dxgi" options.capture = .DXGI;
else if how == "wgc" options.capture = .WGC;
else { print("-capture takes auto, dxgi or wgc\n"); exit(2); }
i += 2; continue;
}
if a == "-check" && more >= 1 { if a == "-check" && more >= 1 {
image, info, ok, error := video_poster(args[i + 1]); image, info, ok, error := video_poster(args[i + 1]);
if !ok { print("%: cannot be read: %\n", args[i + 1], error); exit(1); } if !ok { print("%: cannot be read: %\n", args[i + 1], error); exit(1); }
@ -63,7 +85,13 @@ main :: () {
if !dpi_unaware { if !dpi_unaware {
DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2 :: cast(HANDLE) -4; DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2 :: cast(HANDLE) -4;
if !SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) print("Could not become per-monitor DPI aware (v2).\n"); if !SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) {
error := GetLastError();
ERROR_ACCESS_DENIED :: 5;
if error != ERROR_ACCESS_DENIED print("Could not become per-monitor DPI aware (v2): error %\n", error);
// Access denied: the awareness was set already, as it can be once only. dxgi_open and wgc_open
// refuse when the desktop they are told of is scaled.
}
} }
print("DPI awareness: %\n", ifx dpi_unaware then "none" else "per monitor (v2), as mcap"); print("DPI awareness: %\n", ifx dpi_unaware then "none" else "per monitor (v2), as mcap");
@ -83,6 +111,10 @@ main :: () {
if !ok exit(1); if !ok exit(1);
} }
border_screen := screens[ifx options.screen >= 0 then options.screen else 0].desktop;
border_before: s64;
if border_test border_before = yellow_edge_pixels(border_screen);
recorder: Recorder; recorder: Recorder;
ok, error, fall_back := record_start(*recorder, options); ok, error, fall_back := record_start(*recorder, options);
if !ok { if !ok {
@ -91,17 +123,26 @@ main :: () {
exit(ifx fall_back then cast(s32) 3 else 1); exit(ifx fall_back then cast(s32) 3 else 1);
} }
stats := record_stats(*recorder); stats := record_stats(*recorder);
if stats.fell_back_from print("Desktop Duplication could not run (%), so:\n", stats.fell_back_from);
print("Recording % x % for % s at up to % fps with % and %, to %\n", stats.width, stats.height, seconds, options.fps, stats.source, stats.encoder, options.path); print("Recording % x % for % s at up to % fps with % and %, to %\n", stats.width, stats.height, seconds, options.fps, stats.source, stats.encoder, options.path);
region := options.region; region := options.region;
if region.right <= region.left region = screens[ifx options.screen >= 0 then options.screen else 0].desktop; if region.right <= region.left region = screens[ifx options.screen >= 0 then options.screen else 0].desktop;
excluded, control: HWND; excluded, control: HWND;
if exclude_test excluded, control = exclusion_windows(region); if exclude_test excluded, control = exclusion_windows(region);
saved_cursor: POINT;
cursor_at: POINT;
if cursor_test {
GetCursorPos(*saved_cursor);
cursor_at = .{region.left + (region.right - region.left) / 2, region.top + (region.bottom - region.top) / 2};
SetCursorPos(cursor_at.x, cursor_at.y);
}
mover: HWND; mover: HWND;
if animate mover = square_window("record-probe animate", 0x0000A5FF, region.left, region.top, 120); if animate mover = square_window("record-probe animate", 0x0000A5FF, region.left, region.top, 120);
start := current_time_monotonic(); start := current_time_monotonic();
last_print := start; last_print := start;
border_during: s64 = -1;
while to_float64_seconds(current_time_monotonic() - start) < seconds { while to_float64_seconds(current_time_monotonic() - start) < seconds {
pump_messages(); pump_messages();
if mover { // back and forth across the region, about 240 times a second if mover { // back and forth across the region, about 240 times a second
@ -120,12 +161,15 @@ main :: () {
print(" % s: % captured, % sent (% repeats), % written, % dropped, % lost, % KB\n", print(" % s: % captured, % sent (% repeats), % written, % dropped, % lost, % KB\n",
FF(to_float64_seconds(last_print - start), 0), s.captured, s.emitted, s.repeats, s.written, s.dropped, s.lost, s.bytes / 1024); FF(to_float64_seconds(last_print - start), 0), s.captured, s.emitted, s.repeats, s.written, s.dropped, s.lost, s.bytes / 1024);
} }
if border_test && border_during < 0 && to_float64_seconds(current_time_monotonic() - start) >= 1.5
border_during = yellow_edge_pixels(border_screen);
failed, _ := record_failed(*recorder); failed, _ := record_failed(*recorder);
if failed break; if failed break;
} }
last, width, height := record_last_frame(*recorder); last, width, height := record_last_frame(*recorder);
if mover DestroyWindow(mover); if mover DestroyWindow(mover);
if cursor_test SetCursorPos(saved_cursor.x, saved_cursor.y);
stopped, stop_error := record_stop(*recorder); stopped, stop_error := record_stop(*recorder);
s := recorder.stats; s := recorder.stats;
print("\n%\n", ifx stopped then "Recorded." else tprint("Recording failed: %", stop_error)); print("\n%\n", ifx stopped then "Recorded." else tprint("Recording failed: %", stop_error));
@ -159,9 +203,28 @@ main :: () {
DestroyWindow(control); DestroyWindow(control);
} }
if border_test {
print(" border: % yellow pixels along the screen's edges before recording, % while: %\n", border_before, border_during,
ifx border_during < 0 then "not measured (too short)"
else ifx border_during > border_before + 200 then "a capture border is drawn"
else "no capture border");
}
if snapshot_folder { if snapshot_folder {
make_directory_if_it_does_not_exist(snapshot_folder, recursive = true); make_directory_if_it_does_not_exist(snapshot_folder, recursive = true);
rgba := bgra_to_rgba(last); rgba := bgra_to_rgba(last);
if cursor_test { // 48 x 48 around where the mouse was put, four times as large
SIDE :: 48; SCALE :: 4;
crop := NewArray(SIDE * SCALE * SIDE * SCALE * 4, u8,, temp);
cx, cy := cursor_at.x - region.left - 8, cursor_at.y - region.top - 8;
for y: 0..SIDE * SCALE - 1 for x: 0..SIDE * SCALE - 1 {
sx, sy := clamp(cx + x / SCALE, 0, width - 1), clamp(cy + y / SCALE, 0, height - 1);
memcpy(crop.data + (y * SIDE * SCALE + x) * 4, rgba.data + (cast(s64) sy * width + sx) * 4, 4);
}
cursor_path := tprint("%/cursor.png", snapshot_folder);
stbi_write_png(temp_c_string(cursor_path), SIDE * SCALE, SIDE * SCALE, 4, crop.data, SIDE * SCALE * 4);
print(" cursor: %\n", cursor_path);
}
captured_path := tprint("%/captured.png", snapshot_folder); captured_path := tprint("%/captured.png", snapshot_folder);
stbi_write_png(temp_c_string(captured_path), width, height, 4, rgba.data, width * 4); stbi_write_png(temp_c_string(captured_path), width, height, 4, rgba.data, width * 4);
image, info, decoded, decode_error := video_poster(options.path); image, info, decoded, decode_error := video_poster(options.path);
@ -210,6 +273,40 @@ square_window :: (class_name: string, colour: u32, x: s32, y: s32, side: s32) ->
WS_POPUP | WS_VISIBLE, x, y, side, side, null, null, instance, null); WS_POPUP | WS_VISIBLE, x, y, side, side, null, null, instance, null);
} }
// Pixels along a screen's outer 6 pixels that look like Windows' yellow capture border, with GDI.
yellow_edge_pixels :: (screen: RECT) -> s64 {
width, height := screen.right - screen.left, screen.bottom - screen.top;
desktop := GetDC(null);
defer ReleaseDC(null, desktop);
dc := CreateCompatibleDC(desktop);
defer DeleteDC(dc);
bitmap := CreateCompatibleBitmap(desktop, width, height);
defer DeleteObject(bitmap);
old := SelectObject(dc, bitmap);
BitBlt(dc, 0, 0, width, height, desktop, screen.left, screen.top, cast(ROP) (cast(u32) ROP.SRCCOPY | 0x40000000)); // CAPTUREBLT
SelectObject(dc, old);
info: BITMAPINFO;
info.bmiHeader.biSize = size_of(BITMAPINFOHEADER);
info.bmiHeader.biWidth = width;
info.bmiHeader.biHeight = -height;
info.bmiHeader.biPlanes = 1;
info.bmiHeader.biBitCount = 32;
info.bmiHeader.biCompression = BI_RGB;
pixels := NewArray(cast(s64) width * height * 4, u8,, temp);
GetDIBits(dc, bitmap, 0, xx height, pixels.data, *info, DIB_RGB_COLORS);
EDGE :: 6;
count: s64;
for y: 0..height - 1 for x: 0..width - 1 {
if x >= EDGE && x < width - EDGE && y >= EDGE && y < height - EDGE continue;
p := pixels.data + (cast(s64) y * width + x) * 4;
b, g, r := cast(s32) p[0], cast(s32) p[1], cast(s32) p[2];
if r > 200 && g > 120 && b < 90 && r - b > 130 count += 1;
}
return count;
}
// A window's rectangle, in the region's pixels, a few pixels in from its edges. // A window's rectangle, in the region's pixels, a few pixels in from its edges.
window_rect_in :: (window: HWND, region: RECT) -> RECT { window_rect_in :: (window: HWND, region: RECT) -> RECT {
r: RECT; r: RECT;
@ -287,6 +384,7 @@ FF :: (value: float64, decimals: s64) -> FormatFloat #expand {
SetProcessDpiAwarenessContext :: (value: HANDLE) -> BOOL #foreign user32_dll; SetProcessDpiAwarenessContext :: (value: HANDLE) -> BOOL #foreign user32_dll;
SetWindowDisplayAffinity :: (window: HWND, affinity: u32) -> BOOL #foreign user32_dll; SetWindowDisplayAffinity :: (window: HWND, affinity: u32) -> BOOL #foreign user32_dll;
SetCursorPos :: (x: s32, y: s32) -> BOOL #foreign user32_dll;
user32_dll :: #library,system "user32"; user32_dll :: #library,system "user32";
#import "Basic"; #import "Basic";

34
video-capture-plan.html
View File

@ -68,7 +68,7 @@
<header> <header>
<h1>Screen video capture in mcap</h1> <h1>Screen video capture in mcap</h1>
<p>Planning document · 2026-10-10 · for mcap after 0.1.1 (still captures with PrintScreen) · <strong>phase 1 built, mcap 0.1.3</strong></p> <p>Planning document · 2026-10-10 · for mcap after 0.1.1 (still captures with PrintScreen) · <strong>phases 1 and 2 built, mcap 0.1.4</strong></p>
</header> </header>
<div class="summary"> <div class="summary">
@ -81,6 +81,7 @@
<li><span class="tag decided">Decided</span> <strong>A recording becomes an <code>.mcap</code></strong> in <em>Screen Captures</em>, the folder stills go to. It's stored the way imported videos are: the MP4 as the original, a frame from it for the thumbnail, OCR and labels.</li> <li><span class="tag decided">Decided</span> <strong>A recording becomes an <code>.mcap</code></strong> in <em>Screen Captures</em>, the folder stills go to. It's stored the way imported videos are: the MP4 as the original, a frame from it for the thumbnail, OCR and labels.</li>
<li>It's built in five phases. Phase 1 is a command-line probe that answers the two-GPU question on your laptop before anything depends on it.</li> <li>It's built in five phases. Phase 1 is a command-line probe that answers the two-GPU question on your laptop before anything depends on it.</li>
<li><span class="tag decided">Done</span> <strong>Phase 1</strong> (2026-10-10, mcap 0.1.3): <code>modules/Screen_Record</code> and <code>record-probe</code>. 60 s of 4K at 30 fps with NVENC and no dropped frames. The two-GPU failure couldn't be reproduced, because this laptop's screens are currently driven by the NVIDIA GPU (see <a href="#phases">phase 1's result</a>).</li> <li><span class="tag decided">Done</span> <strong>Phase 1</strong> (2026-10-10, mcap 0.1.3): <code>modules/Screen_Record</code> and <code>record-probe</code>. 60 s of 4K at 30 fps with NVENC and no dropped frames. The two-GPU failure couldn't be reproduced, because this laptop's screens are currently driven by the NVIDIA GPU (see <a href="#phases">phase 1's result</a>).</li>
<li><span class="tag decided">Done</span> <strong>Phase 2</strong> (2026-10-10, mcap 0.1.4): the Windows.Graphics.Capture fallback. Automatic mode falls back to it on a simulated <code>DXGI_ERROR_UNSUPPORTED</code>. It records 4K at 60 fps with nothing dropped, draws the cursor exactly as Desktop Duplication's code does, and the yellow border can be turned off on Windows 11.</li>
<li><span class="tag later">Later</span> Audio is phase 5. Recording one window, and regions that span two screens, come after.</li> <li><span class="tag later">Later</span> Audio is phase 5. Recording one window, and regions that span two screens, come after.</li>
</ul> </ul>
</div> </div>
@ -321,7 +322,7 @@ Captured_Frame :: struct {
</div> </div>
<div class="phase"> <div class="phase">
<h3>Phase 2: Windows.Graphics.Capture fallback</h3> <h3>Phase 2: Windows.Graphics.Capture fallback <span class="tag decided">Done</span></h3>
<p>The second source behind the same interface: WinRT bindings, polling <code>TryGetNextFrame</code>, the cursor and border settings. The fallback rules from 3.2, and <code>-capture dxgi|wgc|auto</code> on the probe.</p> <p>The second source behind the same interface: WinRT bindings, polling <code>TryGetNextFrame</code>, the cursor and border settings. The fallback rules from 3.2, and <code>-capture dxgi|wgc|auto</code> on the probe.</p>
<p><strong>Done when:</strong></p> <p><strong>Done when:</strong></p>
<ul> <ul>
@ -329,6 +330,35 @@ Captured_Frame :: struct {
<li><code>auto</code> picks Desktop Duplication where it works and Windows.Graphics.Capture where it doesn't. The latter is shown either on the two-GPU laptop or on a rotated screen.</li> <li><code>auto</code> picks Desktop Duplication where it works and Windows.Graphics.Capture where it doesn't. The latter is shown either on the two-GPU laptop or on a rotated screen.</li>
<li>Whether the yellow border can be turned off is recorded.</li> <li>Whether the yellow border can be turned off is recorded.</li>
</ul> </ul>
<div class="note"><strong>Result (2026-10-10).</strong> Release build, the same laptop and screens as phase 1.
<ul>
<li><strong>Built:</strong>
<ul>
<li><code>wgc_source.jai</code>: hand-written bindings for eight WinRT interfaces.</li>
<li><code>capture_gpu.jai</code>: what both sources share, namely the device on the screen's adapter, the region copy and the staging ring. Each source is now only "the next image's region into a texture".</li>
<li><code>Record_Options.capture</code>: <code>.AUTO</code>, <code>.DXGI</code> or <code>.WGC</code>.</li>
<li>The probe's <code>-capture auto|dxgi|wgc</code>.</li>
</ul></li>
<li><strong>The fallback:</strong>
<ul>
<li><code>auto</code> picks Desktop Duplication where it works.</li>
<li>With <code>-simulate-unsupported</code>, which makes duplication fail with <code>DXGI_ERROR_UNSUPPORTED</code> as on a two-GPU laptop, <code>auto</code> logs why and records with Windows.Graphics.Capture.</li>
<li><code>-capture dxgi</code> with the same simulated failure refuses, with exit code 3.</li>
<li>The real two-GPU case is shelved: this laptop's GPU is pinned in the BIOS.</li>
</ul></li>
<li><strong>Recording:</strong> 4K at 60 fps with motion: 56.6 frames a second taken and every one encoded, none dropped. Windows.Graphics.Capture delivers a little under the screen's 60 Hz; Desktop Duplication delivered 59.3. A 641 × 481 region on the internal screen gives the same frame as Desktop Duplication's in phase 1.</li>
<li><strong>The border:</strong> <code>IsBorderRequired = false</code> works for mcap as an unpackaged app on Windows 11, with no access request. A GDI sample of the screen's outer 6 pixels found no yellow pixels with the border off, and 14,214 with it left on (<code>-keep-border</code>, the control).</li>
<li><strong>The cursor:</strong> with the mouse put in the region (<code>-cursor-test</code>), Windows.Graphics.Capture's own cursor and the one <code>cursor.jai</code> draws into Desktop Duplication's frames are the same, pixel for pixel, by eye at 4×.</li>
<li><strong><code>WDA_EXCLUDEFROMCAPTURE</code></strong> is respected by Windows.Graphics.Capture too: the excluded square is 0% present and the control square 100%.</li>
<li><strong>Waiting for frames:</strong> each new frame sets an event, through a small COM handler for <code>FrameArrived</code>, which Windows took. The polling fallback, for when Windows refuses the handler (<code>-poll</code>), takes 41 frames a second of a 60 Hz screen, enough for a 30 fps recording (29.0 sent), at a 1 ms timer resolution.</li>
</ul>
<p><strong>Changes from the plan:</strong></p>
<ul>
<li><strong>Events, not polling.</strong> The plan said to poll <code>TryGetNextFrame</code>, to avoid writing a WinRT event handler in Jai. The handler took about 30 lines, so it's the default, and polling is the fallback.</li>
<li><strong>The source opens on the capture thread</strong>, for both methods. WinRT objects belong to the apartment that made them, so <code>record_start</code> starts the capture thread and waits for it to say whether its source opened.</li>
<li><strong>The WinRT calls</strong> (<code>RoInitialize</code>, <code>RoGetActivationFactory</code>, HSTRINGs) are loaded from <code>combase.dll</code> at run time. Their import library, <code>runtimeobject.lib</code>, names no DLL of its own, which Jai's <code>#library</code> needs.</li>
<li><strong>Test switches:</strong> <code>Record_Options.testing</code> holds three, for the probe: simulate the two-GPU failure, force polling, keep the border.</li>
</ul></div>
</div> </div>
<div class="phase"> <div class="phase">