[0.1.4] Video capture phase 2: Windows.Graphics.Capture, for when Desktop Duplication cannot run
- modules/Screen_Record/wgc_source.jai: the screen through Windows.Graphics.Capture, with hand-written bindings for its WinRT interfaces (capture item interop, item, free-threaded frame pool, frame, session 1-3, IDirect3DDxgiInterfaceAccess, IClosable). A small COM handler on FrameArrived sets an event per frame; when Windows refuses it the pool is polled each millisecond. The cursor is drawn by Windows; the yellow border is turned off where the system allows (Windows 11). A screen that changes size makes the pool again. WinRT's own calls are loaded from combase.dll at run time: runtimeobject.lib names no DLL for #library. - capture_gpu.jai: what both sources share (the device on the screen's adapter, the region copy, the staging ring), so each is only "the next image's region into a texture". - Record_Options.capture: AUTO (Desktop Duplication, else Windows.Graphics.Capture when it says fall_back), DXGI or WGC. Sources open on the capture thread, as WinRT objects belong to their apartment; record_start waits for it to say how that went. Record_Options.testing simulates the two-GPU failure, forces polling, or keeps the border, for record-probe. - record-probe: -capture auto|dxgi|wgc, -simulate-unsupported, -poll, -border-test with -keep-border as its control, -cursor-test; DPI awareness set already is no longer reported as a failure. - On the RTX 5080 laptop: auto falls back on the simulated failure; WGC records 4K at 60 fps with nothing dropped; its cursor and the one cursor.jai draws for duplication are the same; the border is off for an unpackaged app (0 yellow edge pixels, 14,214 with it kept); WDA_EXCLUDEFROMCAPTURE is respected; polling holds 30 fps. The plan records the results. - README: Screen_Record's line, which held a carriage return in place of "\r" since 0.1.3.
This commit is contained in:
parent
06a2aacc9c
commit
9953ba6f1f
@ -357,9 +357,11 @@ converted copy.
|
||||
(`video.jai`), and how many frames a GIF has. `bin\image-probe FILE...` (`jai first.jai -
|
||||
image_probe`) shows what it makes of files.
|
||||
- `modules/Screen_Record`: a region of one screen recorded to H.264 in an MP4, with DXGI Desktop
|
||||
Duplication and FFmpeg (NVENC, else AMF, QSV, Media Foundation or libx264), for screen recording
|
||||
(`video-capture-plan.html`). `bin
ecord-probe` (`jai first.jai - record_probe`) records with it
|
||||
and says how it went; `-list` shows the screens, `-check FILE` reads a recording back.
|
||||
Duplication, or Windows.Graphics.Capture where duplication cannot run (two GPUs, a rotated screen),
|
||||
and FFmpeg (NVENC, else AMF, QSV, Media Foundation or libx264), for screen recording
|
||||
(`video-capture-plan.html`). `bin\record-probe` (`jai first.jai - record_probe`) records with it
|
||||
and says how it went; `-list` shows the screens, `-capture auto|dxgi|wgc` picks the way, `-check
|
||||
FILE` reads a recording back.
|
||||
- `modules/Snagx`: `.snagx` to `.mcap`, copied from SnagitConverter.
|
||||
- `modules/Capture_File`: reading and writing `.mcap` files, with a video's original left on disk.
|
||||
- `modules/Segment`: panoptic segmentation with Mask2Former, run with onnxruntime (see its README and
|
||||
|
||||
@ -29,7 +29,7 @@ APP_NAME :: "mcap";
|
||||
APP_FOLDER_NAME :: "mcap";
|
||||
GIT_BRANCH :: "check";
|
||||
GIT_HASH :: "-";
|
||||
VERSION :: "0.1.3";
|
||||
VERSION :: "0.1.4";
|
||||
BUILD_DATE :: "-";
|
||||
DEBUG :: true;
|
||||
COMPILE_TESTS :: true;
|
||||
|
||||
@ -8,7 +8,7 @@ APP_NAME :: "mcap";
|
||||
APP_FOLDER_NAME :: "mcap";
|
||||
APP_BUNDLE_IDENTIFIER :: "com.example.mcap";
|
||||
|
||||
VERSION :: "0.1.3";
|
||||
VERSION :: "0.1.4";
|
||||
// Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated
|
||||
// builds never write over the ones you run by hand.
|
||||
build_directory := "bin";
|
||||
|
||||
105
modules/Screen_Record/capture_gpu.jai
Normal file
105
modules/Screen_Record/capture_gpu.jai
Normal file
@ -0,0 +1,105 @@
|
||||
// What both ways of taking the screen share (dxgi_source.jai, wgc_source.jai): a D3D11 device of
|
||||
// their own on the adapter the screen is attached to, the region as last seen, copied out of each
|
||||
// new image on the GPU, and the staging textures it is read back through.
|
||||
|
||||
Capture_Gpu :: struct {
|
||||
adapter: *IDXGIAdapter1;
|
||||
device: *ID3D11Device;
|
||||
device_context: *ID3D11DeviceContext;
|
||||
box: D3D11_BOX; // the region, in the screen's pixels
|
||||
latest: *ID3D11Texture2D; // the region as last seen
|
||||
staging: [RECORD_STAGING] *ID3D11Texture2D;
|
||||
copy_ticks: s64; // spent taking the region out of images (QueryPerformanceCounter ticks)
|
||||
}
|
||||
|
||||
RECORD_STAGING :: 3; // staging textures in turn: the one read is two frames behind the one copied
|
||||
|
||||
// What a source's next() found.
|
||||
Capture_Next :: enum u8 { NONE; IMAGE; POINTER; IMAGE_AND_POINTER; LOST; FAILED; }
|
||||
|
||||
// The device on the screen's adapter, and the textures for a region (in the screen's pixels).
|
||||
gpu_open :: (g: *Capture_Gpu, screen: Record_Screen, region: RECT) -> ok: bool, error: string {
|
||||
factory: *IDXGIFactory1;
|
||||
hr := CreateDXGIFactory1(*uid(IDXGIFactory1_UUID), xx *factory);
|
||||
if FAILED(hr) return false, hresult_text("CreateDXGIFactory1", hr);
|
||||
defer IUnknown_Release(factory);
|
||||
hr = IDXGIFactory1_EnumAdapters1(factory, screen.adapter_index, *g.adapter);
|
||||
if FAILED(hr) return false, hresult_text("the screen's adapter is gone", hr);
|
||||
|
||||
levels := D3D_FEATURE_LEVEL.[._11_1, ._11_0, ._10_1, ._10_0];
|
||||
level: D3D_FEATURE_LEVEL;
|
||||
hr = D3D11CreateDevice(g.adapter, .UNKNOWN, null, .BGRA_SUPPORT, levels.data, levels.count, D3D11_SDK_VERSION, *g.device, *level, *g.device_context);
|
||||
if FAILED(hr) return false, hresult_text("D3D11CreateDevice on the screen's adapter", hr);
|
||||
|
||||
g.box = .{left = xx region.left, top = xx region.top, right = xx region.right, bottom = xx region.bottom, front = 0, back = 1};
|
||||
desc: D3D11_TEXTURE2D_DESC;
|
||||
desc.Width = cast(u32) (region.right - region.left);
|
||||
desc.Height = cast(u32) (region.bottom - region.top);
|
||||
desc.MipLevels = 1;
|
||||
desc.ArraySize = 1;
|
||||
desc.Format = .B8G8R8A8_UNORM;
|
||||
desc.SampleDesc.Count = 1;
|
||||
desc.Usage = .DEFAULT;
|
||||
hr = ID3D11Device_CreateTexture2D(g.device, *desc, null, *g.latest);
|
||||
if FAILED(hr) return false, hresult_text("a texture for the region", hr);
|
||||
desc.Usage = .STAGING;
|
||||
desc.CPUAccessFlags = .READ;
|
||||
for * g.staging {
|
||||
hr = ID3D11Device_CreateTexture2D(g.device, *desc, null, it);
|
||||
if FAILED(hr) return false, hresult_text("a staging texture", hr);
|
||||
}
|
||||
return true, "";
|
||||
}
|
||||
|
||||
// The region of a whole screen's image into g.latest.
|
||||
gpu_copy_region :: (g: *Capture_Gpu, image: *ID3D11Texture2D) {
|
||||
start := ticks_now();
|
||||
ID3D11DeviceContext_CopySubresourceRegion(g.device_context, g.latest, 0, 0, 0, 0, image, 0, *g.box);
|
||||
g.copy_ticks += ticks_now() - start;
|
||||
}
|
||||
|
||||
gpu_close :: (g: *Capture_Gpu) {
|
||||
for * g.staging com_release(it);
|
||||
com_release(*g.latest);
|
||||
com_release(*g.device_context);
|
||||
com_release(*g.device);
|
||||
com_release(*g.adapter);
|
||||
}
|
||||
|
||||
#scope_module
|
||||
|
||||
com_release :: (p: *$T) {
|
||||
if p.* { IUnknown_Release(cast(*IUnknown) p.*); p.* = null; }
|
||||
}
|
||||
|
||||
ticks_now :: () -> s64 {
|
||||
t: LARGE_INTEGER;
|
||||
QueryPerformanceCounter(*t);
|
||||
return t.QuadPart;
|
||||
}
|
||||
|
||||
hresult_is :: (hr: HRESULT, code: u32) -> bool {
|
||||
return cast,no_check(u32) hr == code;
|
||||
}
|
||||
|
||||
// "what failed: 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", temp.
|
||||
hresult_text :: (what: string, hr: HRESULT) -> string {
|
||||
code := cast,no_check(u32) hr;
|
||||
name := "";
|
||||
if code == {
|
||||
case DXGI_ERROR_UNSUPPORTED; name = " (DXGI_ERROR_UNSUPPORTED)";
|
||||
case DXGI_ERROR_NOT_CURRENTLY_AVAILABLE; name = " (DXGI_ERROR_NOT_CURRENTLY_AVAILABLE)";
|
||||
case DXGI_ERROR_ACCESS_LOST; name = " (DXGI_ERROR_ACCESS_LOST)";
|
||||
case 0x80070005; name = " (E_ACCESSDENIED)";
|
||||
case 0x80004002; name = " (E_NOINTERFACE)";
|
||||
case 0x80070057; name = " (E_INVALIDARG)";
|
||||
case 0x80040154; name = " (REGDB_E_CLASSNOTREG)";
|
||||
case 0x80004001; name = " (E_NOTIMPL)";
|
||||
}
|
||||
return tprint("%: 0x%%", what, formatInt(code, base = 16, minimum_digits = 8), name);
|
||||
}
|
||||
|
||||
wide_text :: (wide: *s16) -> string {
|
||||
text, ok := wide_to_utf8(cast(*u16) wide,, temp);
|
||||
return ifx ok then text else "";
|
||||
}
|
||||
@ -1,37 +1,29 @@
|
||||
// DXGI Desktop Duplication: the screens as the GPU composes them, a frame each time one changes.
|
||||
// DXGI Desktop Duplication: the screens as the GPU composes them, a frame each time one changes,
|
||||
// with the cursor apart (cursor.jai draws it in).
|
||||
//
|
||||
// It runs on a D3D11 device of its own, made on the adapter the screen is attached to: duplicating
|
||||
// It runs on a device made on the adapter the screen is attached to (capture_gpu.jai): duplicating
|
||||
// with a device on another adapter fails. On a laptop with two GPUs, where the screen hangs off one
|
||||
// and the process may run on the other, DuplicateOutput1 can fail with DXGI_ERROR_UNSUPPORTED all the
|
||||
// same; that, too many duplications of the screen at once, and a rotated screen are what
|
||||
// dxgi_open returns fall_back for.
|
||||
// same; that, too many duplications of the screen at once, and a rotated screen are what dxgi_open
|
||||
// returns fall_back for, and Windows.Graphics.Capture (wgc_source.jai) takes over.
|
||||
//
|
||||
// DuplicateOutput1 (IDXGIOutput5) is used where there is one: the plain DuplicateOutput is documented
|
||||
// to fail for a process aware of each monitor's DPI, as mcap is. legacy forces the plain one, for
|
||||
// record-probe to check that.
|
||||
// to fail for a process aware of each monitor's DPI, as mcap is (it did not on an RTX 5080 laptop).
|
||||
// legacy forces the plain one, for record-probe.
|
||||
|
||||
Dxgi_Source :: struct {
|
||||
adapter: *IDXGIAdapter1;
|
||||
#as using gpu: Capture_Gpu;
|
||||
output: *IDXGIOutput1;
|
||||
output5: *IDXGIOutput5; // null before Windows 10 1703
|
||||
device: *ID3D11Device;
|
||||
device_context: *ID3D11DeviceContext;
|
||||
duplication: *IDXGIOutputDuplication;
|
||||
legacy: bool;
|
||||
method: string; // which duplication call worked, for the log
|
||||
|
||||
box: D3D11_BOX; // the region, in the output's pixels
|
||||
latest: *ID3D11Texture2D; // the region as last seen
|
||||
staging: [RECORD_STAGING] *ID3D11Texture2D;
|
||||
|
||||
copy_ticks: s64; // spent taking images out of frames: QueryInterface and the copy's submission
|
||||
pointer_position: POINT; // the cursor's top left, in the output's pixels
|
||||
pointer_position: POINT; // the cursor's top left, in the screen's pixels
|
||||
pointer_visible: bool;
|
||||
shape: Cursor_Shape;
|
||||
}
|
||||
|
||||
RECORD_STAGING :: 3; // staging textures in turn: the one read is two frames behind the one copied
|
||||
|
||||
// Every screen attached to the desktop, on every adapter, in DXGI's order. Temp.
|
||||
record_list_screens :: () -> [] Record_Screen {
|
||||
screens: [..] Record_Screen;
|
||||
@ -58,7 +50,8 @@ record_list_screens :: () -> [] Record_Screen {
|
||||
IUnknown_Release(output);
|
||||
if desc.AttachedToDesktop != .FALSE {
|
||||
array_add(*screens, .{adapter_index = adapter_index, output_index = output_index, adapter = adapter_name,
|
||||
name = wide_text(desc.DeviceName.data), desktop = desc.DesktopCoordinates, rotation = desc.Rotation});
|
||||
name = wide_text(desc.DeviceName.data), desktop = desc.DesktopCoordinates,
|
||||
rotation = desc.Rotation, monitor = desc.Monitor});
|
||||
}
|
||||
output_index += 1;
|
||||
}
|
||||
@ -67,51 +60,22 @@ record_list_screens :: () -> [] Record_Screen {
|
||||
return screens;
|
||||
}
|
||||
|
||||
// Makes the device on the screen's adapter, and duplicates the screen. region is in the screen's
|
||||
// pixels and lies on it.
|
||||
// Duplicates the screen. region is in the screen's pixels and lies on it.
|
||||
dxgi_open :: (s: *Dxgi_Source, screen: Record_Screen, region: RECT, legacy: bool) -> ok: bool, error: string, fall_back: bool {
|
||||
s.legacy = legacy;
|
||||
if screen.rotation != .IDENTITY && screen.rotation != .UNSPECIFIED return false, "the screen is rotated", true;
|
||||
|
||||
factory: *IDXGIFactory1;
|
||||
hr := CreateDXGIFactory1(*uid(IDXGIFactory1_UUID), xx *factory);
|
||||
if FAILED(hr) return false, hresult_text("CreateDXGIFactory1", hr), false;
|
||||
defer IUnknown_Release(factory);
|
||||
hr = IDXGIFactory1_EnumAdapters1(factory, screen.adapter_index, *s.adapter);
|
||||
if FAILED(hr) return false, hresult_text("the screen's adapter is gone", hr), false;
|
||||
ok, error := gpu_open(s, screen, region);
|
||||
if !ok return false, error, false;
|
||||
|
||||
output: *IDXGIOutput;
|
||||
hr = IDXGIAdapter_EnumOutputs(s.adapter, screen.output_index, *output);
|
||||
hr := IDXGIAdapter_EnumOutputs(s.adapter, screen.output_index, *output);
|
||||
if FAILED(hr) return false, hresult_text("the screen is gone", hr), false;
|
||||
defer IUnknown_Release(output);
|
||||
hr = IUnknown_QueryInterface(output, *uid(IDXGIOutput1_UUID), xx *s.output);
|
||||
if FAILED(hr) return false, hresult_text("the screen cannot be duplicated (no IDXGIOutput1)", hr), true;
|
||||
IUnknown_QueryInterface(output, *uid(IDXGIOutput5_UUID), xx *s.output5); // optional
|
||||
|
||||
levels := D3D_FEATURE_LEVEL.[._11_1, ._11_0, ._10_1, ._10_0];
|
||||
level: D3D_FEATURE_LEVEL;
|
||||
hr = D3D11CreateDevice(s.adapter, .UNKNOWN, null, .BGRA_SUPPORT, levels.data, levels.count, D3D11_SDK_VERSION, *s.device, *level, *s.device_context);
|
||||
if FAILED(hr) return false, hresult_text("D3D11CreateDevice on the screen's adapter", hr), true;
|
||||
|
||||
s.box = .{left = xx region.left, top = xx region.top, right = xx region.right, bottom = xx region.bottom, front = 0, back = 1};
|
||||
width := cast(u32) (region.right - region.left);
|
||||
height := cast(u32) (region.bottom - region.top);
|
||||
desc: D3D11_TEXTURE2D_DESC;
|
||||
desc.Width = width;
|
||||
desc.Height = height;
|
||||
desc.MipLevels = 1;
|
||||
desc.ArraySize = 1;
|
||||
desc.Format = .B8G8R8A8_UNORM;
|
||||
desc.SampleDesc.Count = 1;
|
||||
desc.Usage = .DEFAULT;
|
||||
hr = ID3D11Device_CreateTexture2D(s.device, *desc, null, *s.latest);
|
||||
if FAILED(hr) return false, hresult_text("a texture for the region", hr), false;
|
||||
desc.Usage = .STAGING;
|
||||
desc.CPUAccessFlags = .READ;
|
||||
for * s.staging {
|
||||
hr = ID3D11Device_CreateTexture2D(s.device, *desc, null, it);
|
||||
if FAILED(hr) return false, hresult_text("a staging texture", hr), false;
|
||||
}
|
||||
|
||||
hr = dxgi_duplicate(s);
|
||||
if FAILED(hr) {
|
||||
// Unsupported: two GPUs, the device on the wrong one. Not currently available: too many duplications.
|
||||
@ -134,7 +98,7 @@ dxgi_open :: (s: *Dxgi_Source, screen: Record_Screen, region: RECT, legacy: bool
|
||||
|
||||
// Duplicates the screen, again after the duplication was lost.
|
||||
dxgi_duplicate :: (s: *Dxgi_Source) -> HRESULT {
|
||||
if s.duplication { IUnknown_Release(s.duplication); s.duplication = null; }
|
||||
com_release(*s.duplication);
|
||||
if s.output5 && !s.legacy {
|
||||
s.method = "DuplicateOutput1";
|
||||
formats := DXGI_FORMAT.[.B8G8R8A8_UNORM];
|
||||
@ -144,11 +108,9 @@ dxgi_duplicate :: (s: *Dxgi_Source) -> HRESULT {
|
||||
return IDXGIOutput1_DuplicateOutput(s.output, s.device, *s.duplication);
|
||||
}
|
||||
|
||||
Dxgi_Next :: enum u8 { NONE; IMAGE; POINTER; IMAGE_AND_POINTER; LOST; FAILED; }
|
||||
|
||||
// Waits up to timeout_ms for the screen to change. A new image is copied into s.latest; a moved or
|
||||
// changed cursor updates s.pointer_*. On LOST the duplication is gone: call dxgi_duplicate.
|
||||
dxgi_next :: (s: *Dxgi_Source, timeout_ms: u32) -> Dxgi_Next, error: string {
|
||||
// Waits up to timeout_ms for the screen to change. A new image's region goes into s.latest; a moved
|
||||
// or changed cursor updates s.pointer_* and s.shape. On LOST the duplication is gone: dxgi_duplicate.
|
||||
dxgi_next :: (s: *Dxgi_Source, timeout_ms: u32) -> Capture_Next, error: string {
|
||||
info: DXGI_OUTDUPL_FRAME_INFO;
|
||||
resource: *IDXGIResource;
|
||||
hr := IDXGIOutputDuplication_AcquireNextFrame(s.duplication, timeout_ms, *info, *resource);
|
||||
@ -160,16 +122,11 @@ dxgi_next :: (s: *Dxgi_Source, timeout_ms: u32) -> Dxgi_Next, error: string {
|
||||
|
||||
image := info.LastPresentTime.QuadPart != 0;
|
||||
if image {
|
||||
start: LARGE_INTEGER;
|
||||
QueryPerformanceCounter(*start);
|
||||
texture: *ID3D11Texture2D;
|
||||
hr = IUnknown_QueryInterface(resource, *uid(ID3D11Texture2D_UUID), xx *texture);
|
||||
if FAILED(hr) return .FAILED, hresult_text("the frame is not a texture", hr);
|
||||
ID3D11DeviceContext_CopySubresourceRegion(s.device_context, s.latest, 0, 0, 0, 0, texture, 0, *s.box);
|
||||
gpu_copy_region(s, texture);
|
||||
IUnknown_Release(texture);
|
||||
end: LARGE_INTEGER;
|
||||
QueryPerformanceCounter(*end);
|
||||
s.copy_ticks += end.QuadPart - start.QuadPart;
|
||||
}
|
||||
|
||||
pointer := info.LastMouseUpdateTime.QuadPart != 0;
|
||||
@ -198,41 +155,10 @@ dxgi_next :: (s: *Dxgi_Source, timeout_ms: u32) -> Dxgi_Next, error: string {
|
||||
}
|
||||
|
||||
dxgi_close :: (s: *Dxgi_Source) {
|
||||
release :: (p: *$T) { if p.* { IUnknown_Release(p.*); p.* = null; } }
|
||||
release(*s.duplication);
|
||||
for * s.staging release(it);
|
||||
release(*s.latest);
|
||||
release(*s.device_context);
|
||||
release(*s.device);
|
||||
release(*s.output5);
|
||||
release(*s.output);
|
||||
release(*s.adapter);
|
||||
com_release(*s.duplication);
|
||||
com_release(*s.output5);
|
||||
com_release(*s.output);
|
||||
gpu_close(s);
|
||||
array_free(s.shape.data);
|
||||
s.shape = .{};
|
||||
}
|
||||
|
||||
#scope_module
|
||||
|
||||
hresult_is :: (hr: HRESULT, code: u32) -> bool {
|
||||
return cast,no_check(u32) hr == code;
|
||||
}
|
||||
|
||||
// "what failed: 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", temp.
|
||||
hresult_text :: (what: string, hr: HRESULT) -> string {
|
||||
code := cast,no_check(u32) hr;
|
||||
name := "";
|
||||
if code == {
|
||||
case DXGI_ERROR_UNSUPPORTED; name = " (DXGI_ERROR_UNSUPPORTED)";
|
||||
case DXGI_ERROR_NOT_CURRENTLY_AVAILABLE; name = " (DXGI_ERROR_NOT_CURRENTLY_AVAILABLE)";
|
||||
case DXGI_ERROR_ACCESS_LOST; name = " (DXGI_ERROR_ACCESS_LOST)";
|
||||
case 0x80070005; name = " (E_ACCESSDENIED)";
|
||||
case 0x80004002; name = " (E_NOINTERFACE)";
|
||||
case 0x80070057; name = " (E_INVALIDARG)";
|
||||
}
|
||||
return tprint("%: 0x%%", what, formatInt(code, base = 16, minimum_digits = 8), name);
|
||||
}
|
||||
|
||||
wide_text :: (wide: *s16) -> string {
|
||||
text, ok := wide_to_utf8(cast(*u16) wide,, temp);
|
||||
return ifx ok then text else "";
|
||||
}
|
||||
|
||||
@ -2,10 +2,12 @@
|
||||
// recording and tools/record_probe.jai (video-capture-plan.html). Windows only.
|
||||
//
|
||||
// Two threads. The capture thread takes the screen with DXGI Desktop Duplication (dxgi_source.jai),
|
||||
// on a D3D11 device of its own, copies the region on the GPU, reads it back through a ring of staging
|
||||
// textures, so Map never waits on the GPU, and draws the cursor into it (cursor.jai). The encoder
|
||||
// thread hands the frames to FFmpeg (encoder.jai). Between them is a pool of RECORD_BUFFERS frames;
|
||||
// when the encoder falls RECORD_QUEUE frames behind, the oldest waiting is dropped and counted.
|
||||
// or with Windows.Graphics.Capture (wgc_source.jai) where duplication cannot run, each on a D3D11
|
||||
// device of its own; copies the region on the GPU; reads it back through a ring of staging textures,
|
||||
// so Map never waits on the GPU (capture_gpu.jai); and, for duplication, draws the cursor in
|
||||
// (cursor.jai). The encoder thread hands the frames to FFmpeg (encoder.jai). Between them is a pool of
|
||||
// RECORD_BUFFERS frames; when the encoder falls RECORD_QUEUE frames behind, the oldest waiting is
|
||||
// dropped and counted.
|
||||
//
|
||||
// A frame is taken at most fps times a second, and only when the screen or the cursor changed, with
|
||||
// one a second while nothing does, so players can seek. Each carries the time it was taken.
|
||||
@ -15,9 +17,9 @@
|
||||
// ...
|
||||
// ok, error = record_stop(*recorder);
|
||||
//
|
||||
// fall_back says Desktop Duplication cannot run here: on a laptop with two GPUs, on a rotated screen,
|
||||
// or with too many duplications of the screen running. Windows.Graphics.Capture takes over then
|
||||
// (phase 2 of the plan).
|
||||
// With capture .AUTO (the default), Desktop Duplication is tried first and Windows.Graphics.Capture
|
||||
// takes over when it cannot run: on a laptop with two GPUs, on a rotated screen, or with too many
|
||||
// duplications of the screen running. fall_back is only returned for .DXGI, which does not fall back.
|
||||
|
||||
RECORD_MIN_WIDTH :: 240; // a smaller region is refused
|
||||
RECORD_MIN_HEIGHT :: 180;
|
||||
@ -30,6 +32,20 @@ Record_Screen :: struct {
|
||||
name: string; // "\\.\DISPLAY1"
|
||||
desktop: RECT; // on the virtual desktop
|
||||
rotation: DXGI_MODE_ROTATION;
|
||||
monitor: HMONITOR;
|
||||
}
|
||||
|
||||
Record_Capture :: enum u8 {
|
||||
AUTO; // Desktop Duplication, else Windows.Graphics.Capture
|
||||
DXGI; // Desktop Duplication only
|
||||
WGC; // Windows.Graphics.Capture only
|
||||
}
|
||||
|
||||
// For record-probe: paths a machine may not take by itself.
|
||||
Record_Testing :: enum_flags u8 {
|
||||
DXGI_UNSUPPORTED; // Desktop Duplication fails as on a laptop with two GPUs (DXGI_ERROR_UNSUPPORTED)
|
||||
WGC_POLL; // Windows.Graphics.Capture polled, as when it will not take the FrameArrived handler
|
||||
KEEP_BORDER; // Windows.Graphics.Capture's yellow border left on
|
||||
}
|
||||
|
||||
Record_Options :: struct {
|
||||
@ -39,15 +55,18 @@ Record_Options :: struct {
|
||||
fps: s32 = 30;
|
||||
cursor := true;
|
||||
encoder: string; // "" tries RECORD_ENCODERS in turn
|
||||
capture: Record_Capture;
|
||||
legacy_duplicate: bool; // the plain DuplicateOutput, not DuplicateOutput1: for record-probe
|
||||
testing: Record_Testing;
|
||||
}
|
||||
|
||||
Record_Stats :: struct {
|
||||
source: string; // "DXGI DuplicateOutput1 on <adapter>"
|
||||
source: string; // "DXGI DuplicateOutput1 on <adapter>", "Windows.Graphics.Capture on <adapter>, border off"
|
||||
fell_back_from: string; // why Desktop Duplication could not run, when it fell back
|
||||
encoder: string;
|
||||
width, height: s32;
|
||||
captured: s64; // new images of the screen
|
||||
pointer_moves: s64; // cursor changes with no new image
|
||||
pointer_moves: s64; // cursor changes with no new image (Desktop Duplication)
|
||||
emitted: s64; // frames sent to be encoded
|
||||
repeats: s64; // of them, the once-a-second repeat of an unchanged screen
|
||||
written: s64; // encoded
|
||||
@ -60,17 +79,26 @@ Record_Stats :: struct {
|
||||
|
||||
Recorder :: struct {
|
||||
options: Record_Options;
|
||||
source: Dxgi_Source;
|
||||
screen: Record_Screen;
|
||||
local: RECT; // the region, in the screen's pixels
|
||||
dxgi: Dxgi_Source; // the capture thread's: one of these is open
|
||||
wgc: Wgc_Source;
|
||||
using_wgc: bool;
|
||||
encoder: Encoder;
|
||||
width, height: s32;
|
||||
frequency: s64; // QueryPerformanceCounter's
|
||||
start_ticks: s64;
|
||||
|
||||
capture_thread, encoder_thread: Thread;
|
||||
capture_started, encoder_started: bool;
|
||||
running: bool;
|
||||
stopping: bool; // set by record_stop; the capture thread ends, then the encoder
|
||||
capture_ended: bool;
|
||||
|
||||
opened: Semaphore; // the capture thread has opened its source, or failed to
|
||||
open_ok, open_fall_back: bool;
|
||||
open_error: string;
|
||||
|
||||
mutex: Mutex;
|
||||
ready: Semaphore; // signalled for each frame queued, and once when capture ends
|
||||
frames: [RECORD_BUFFERS] Record_Frame;
|
||||
@ -107,48 +135,61 @@ record_start :: (r: *Recorder, options: Record_Options) -> ok: bool, error: stri
|
||||
if screen_index < 0 return false, "the region is not on one screen", false;
|
||||
}
|
||||
if screen_index >= screens.count return false, tprint("there is no screen %", screen_index), false;
|
||||
screen := screens[screen_index];
|
||||
if !has_region region = screen.desktop;
|
||||
if !rect_inside(region, screen.desktop) return false, "the region is not on that screen", false;
|
||||
r.screen = screens[screen_index];
|
||||
r.screen.adapter = copy_string(r.screen.adapter);
|
||||
r.screen.name = copy_string(r.screen.name);
|
||||
if !has_region region = r.screen.desktop;
|
||||
if !rect_inside(region, r.screen.desktop) return false, "the region is not on that screen", false;
|
||||
|
||||
// The minimum first, then even both ways, in the screen's pixels: H.264's 4:2:0 colour needs it.
|
||||
// RECORD_MIN_WIDTH and _HEIGHT are even, so what passes stays at least that size.
|
||||
#assert RECORD_MIN_WIDTH % 2 == 0 && RECORD_MIN_HEIGHT % 2 == 0;
|
||||
if region.right - region.left < RECORD_MIN_WIDTH || region.bottom - region.top < RECORD_MIN_HEIGHT
|
||||
return false, tprint("% x % is smaller than % x %", region.right - region.left, region.bottom - region.top, RECORD_MIN_WIDTH, RECORD_MIN_HEIGHT), false;
|
||||
local := RECT.{region.left - screen.desktop.left, region.top - screen.desktop.top, region.right - screen.desktop.left, region.bottom - screen.desktop.top};
|
||||
local.right = local.left + ((local.right - local.left) & cast(s32) ~1);
|
||||
local.bottom = local.top + ((local.bottom - local.top) & cast(s32) ~1);
|
||||
r.width = local.right - local.left;
|
||||
r.height = local.bottom - local.top;
|
||||
|
||||
ok, error, fall_back := dxgi_open(*r.source, screen, local, options.legacy_duplicate);
|
||||
if !ok { dxgi_close(*r.source); return false, error, fall_back; }
|
||||
|
||||
tried: string;
|
||||
ok, error, tried = encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder);
|
||||
if tried log("[record] Encoders that did not open: %", tried);
|
||||
if !ok { dxgi_close(*r.source); return false, error, false; }
|
||||
|
||||
r.stats.source = sprint("DXGI % on %", r.source.method, screen.adapter);
|
||||
r.stats.encoder = copy_string(r.encoder.name);
|
||||
d := r.screen.desktop;
|
||||
r.local = .{region.left - d.left, region.top - d.top, region.right - d.left, region.bottom - d.top};
|
||||
r.local.right = r.local.left + ((r.local.right - r.local.left) & cast(s32) ~1);
|
||||
r.local.bottom = r.local.top + ((r.local.bottom - r.local.top) & cast(s32) ~1);
|
||||
r.width = r.local.right - r.local.left;
|
||||
r.height = r.local.bottom - r.local.top;
|
||||
r.stats.width, r.stats.height = r.width, r.height;
|
||||
|
||||
init(*r.mutex, "record");
|
||||
init(*r.ready);
|
||||
init(*r.opened);
|
||||
for * r.frames it.pixels = NewArray(cast(s64) r.width * r.height * 4, u8, initialized = false);
|
||||
for 0..RECORD_BUFFERS - 1 r.free[it] = cast(s32) it;
|
||||
r.free_count = RECORD_BUFFERS;
|
||||
r.last_frame = NewArray(cast(s64) r.width * r.height * 4, u8);
|
||||
|
||||
frequency: LARGE_INTEGER;
|
||||
QueryPerformanceFrequency(*frequency);
|
||||
r.frequency = frequency.QuadPart;
|
||||
r.start_ticks = ticks_now();
|
||||
|
||||
// The source is opened on the capture thread, which uses it: WinRT's objects belong to the
|
||||
// apartment they are made in.
|
||||
thread_init(*r.capture_thread, record_capture_proc, name = "record capture");
|
||||
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
|
||||
r.running = true;
|
||||
r.capture_started = true;
|
||||
thread_start(*r.capture_thread, r);
|
||||
wait_for(*r.opened);
|
||||
if !r.open_ok {
|
||||
error, fall_back := copy_string(r.open_error,, temp), r.open_fall_back;
|
||||
record_free(r);
|
||||
return false, error, fall_back;
|
||||
}
|
||||
|
||||
ok, error, tried := encoder_open(*r.encoder, r.options.path, r.width, r.height, r.options.fps, r.options.encoder);
|
||||
if tried log("[record] Encoders that did not open: %", tried);
|
||||
if !ok {
|
||||
error = copy_string(error,, temp);
|
||||
record_free(r);
|
||||
return false, error, false;
|
||||
}
|
||||
r.stats.encoder = copy_string(r.encoder.name);
|
||||
|
||||
thread_init(*r.encoder_thread, record_encoder_proc, name = "record encoder");
|
||||
r.encoder_started = true;
|
||||
r.running = true;
|
||||
thread_start(*r.encoder_thread, r);
|
||||
return true, "", false;
|
||||
}
|
||||
@ -160,20 +201,16 @@ record_stop :: (r: *Recorder) -> ok: bool, error: string {
|
||||
atomic_swap(*r.stopping, true);
|
||||
thread_is_done(*r.capture_thread, -1);
|
||||
thread_is_done(*r.encoder_thread, -1);
|
||||
thread_deinit(*r.capture_thread);
|
||||
thread_deinit(*r.encoder_thread);
|
||||
r.running = false;
|
||||
|
||||
closed, close_error := encoder_close(*r.encoder);
|
||||
r.stats.seconds = cast(float64) (ticks_now() - r.start_ticks) / r.frequency;
|
||||
r.stats.bytes = r.encoder.bytes;
|
||||
dxgi_close(*r.source);
|
||||
for * r.frames array_free(it.pixels);
|
||||
destroy(*r.ready);
|
||||
destroy(*r.mutex);
|
||||
error := ifx r.error then copy_string(r.error,, temp) else "";
|
||||
record_free(r);
|
||||
|
||||
if r.error return false, r.error;
|
||||
if !closed return false, close_error;
|
||||
if error return false, error;
|
||||
if !closed return false, close_error;
|
||||
return true, "";
|
||||
}
|
||||
|
||||
@ -204,9 +241,64 @@ record_failed :: (r: *Recorder) -> bool, string {
|
||||
|
||||
Staged :: struct { pending: bool; ticks: s64; pointer: POINT; pointer_visible: bool; }
|
||||
|
||||
// Stops the capture thread if it runs, and frees what record_start made. The stats' strings are kept.
|
||||
record_free :: (r: *Recorder) {
|
||||
atomic_swap(*r.stopping, true);
|
||||
if r.capture_started { thread_is_done(*r.capture_thread, -1); thread_deinit(*r.capture_thread); r.capture_started = false; }
|
||||
if r.encoder_started { thread_is_done(*r.encoder_thread, -1); thread_deinit(*r.encoder_thread); r.encoder_started = false; }
|
||||
if r.encoder.format encoder_close(*r.encoder);
|
||||
for * r.frames array_free(it.pixels);
|
||||
array_free(r.last_frame);
|
||||
r.last_frame = .[];
|
||||
destroy(*r.opened);
|
||||
destroy(*r.ready);
|
||||
destroy(*r.mutex);
|
||||
free(r.open_error);
|
||||
free(r.error);
|
||||
r.open_error, r.error = "", "";
|
||||
}
|
||||
|
||||
// Desktop Duplication, unless asked for Windows.Graphics.Capture, which takes over when duplication
|
||||
// cannot run here and the options allow.
|
||||
record_open_source :: (r: *Recorder) -> bool {
|
||||
if r.options.capture != .WGC {
|
||||
ok, error, fall_back := dxgi_open(*r.dxgi, r.screen, r.local, r.options.legacy_duplicate);
|
||||
if ok && r.options.testing & .DXGI_UNSUPPORTED {
|
||||
ok, error, fall_back = false, "Desktop Duplication (testing): 0x887A0004 (DXGI_ERROR_UNSUPPORTED)", true;
|
||||
}
|
||||
if ok {
|
||||
r.stats.source = sprint("DXGI % on %", r.dxgi.method, r.screen.adapter);
|
||||
return true;
|
||||
}
|
||||
dxgi_close(*r.dxgi);
|
||||
if r.options.capture == .DXGI || !fall_back {
|
||||
r.open_error, r.open_fall_back = copy_string(error), fall_back;
|
||||
return false;
|
||||
}
|
||||
log("[record] Desktop Duplication cannot run here (%); Windows.Graphics.Capture instead.", error);
|
||||
r.stats.fell_back_from = copy_string(error);
|
||||
}
|
||||
ok, error := wgc_open(*r.wgc, r.screen, r.local, r.options.cursor, r.options.testing);
|
||||
if !ok {
|
||||
wgc_close(*r.wgc);
|
||||
r.open_error = copy_string(error);
|
||||
return false;
|
||||
}
|
||||
r.using_wgc = true;
|
||||
r.stats.source = sprint("Windows.Graphics.Capture on %, %%", r.screen.adapter, r.wgc.border, ifx r.wgc.polling then ", polled" else "");
|
||||
return true;
|
||||
}
|
||||
|
||||
record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
r := cast(*Recorder) thread.data;
|
||||
s := *r.source;
|
||||
wgc_thread_start();
|
||||
defer wgc_thread_end();
|
||||
r.open_ok = record_open_source(r);
|
||||
signal(*r.opened);
|
||||
if !r.open_ok return 0;
|
||||
gpu: *Capture_Gpu = ifx r.using_wgc then *r.wgc.gpu else *r.dxgi.gpu;
|
||||
defer if r.using_wgc wgc_close(*r.wgc); else dxgi_close(*r.dxgi);
|
||||
|
||||
// A change is sent as soon as it comes once its turn is due: turns are a frame's time apart on a
|
||||
// schedule that moves on a frame at a time, so the rate never passes fps, and a tenth of a frame
|
||||
// early counts as on time, so a 60 Hz screen's frames, which jitter, all make a 60 fps cap.
|
||||
@ -221,8 +313,11 @@ record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
while !atomic_read(*r.stopping) {
|
||||
now := ticks_now();
|
||||
due := ifx changed then turn - early else last_emit + r.frequency; // the change, or the repeat
|
||||
wait := clamp((due - now) * 1000 / r.frequency, 0, 100);
|
||||
next, error := dxgi_next(s, cast(u32) wait);
|
||||
wait := cast(u32) clamp((due - now) * 1000 / r.frequency, 0, 100);
|
||||
next: Capture_Next;
|
||||
error: string;
|
||||
if r.using_wgc next, error = wgc_next(*r.wgc, wait);
|
||||
else next, error = dxgi_next(*r.dxgi, wait);
|
||||
if next == {
|
||||
case .FAILED;
|
||||
record_fail(r, error);
|
||||
@ -231,7 +326,7 @@ record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
// A mode change, the secure desktop, a game going fullscreen: duplicated again, and on.
|
||||
lock(*r.mutex); r.stats.lost += 1; unlock(*r.mutex);
|
||||
while !atomic_read(*r.stopping) {
|
||||
hr := dxgi_duplicate(s);
|
||||
hr := dxgi_duplicate(*r.dxgi);
|
||||
if SUCCEEDED(hr) break;
|
||||
if !hresult_is(hr, DXGI_ERROR_ACCESS_LOST) && !hresult_is(hr, 0x80070005) && !hresult_is(hr, DXGI_ERROR_NOT_CURRENTLY_AVAILABLE) {
|
||||
record_fail(r, hresult_text("the screen could not be duplicated again", hr));
|
||||
@ -246,7 +341,7 @@ record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
have_image, changed = true, true;
|
||||
lock(*r.mutex);
|
||||
r.stats.captured += 1;
|
||||
r.stats.copy_ticks = s.copy_ticks;
|
||||
r.stats.copy_ticks = gpu.copy_ticks;
|
||||
unlock(*r.mutex);
|
||||
case .POINTER;
|
||||
if r.options.cursor && have_image {
|
||||
@ -263,9 +358,10 @@ record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
|
||||
// Copied to staging now; read back two frames on, when the GPU is done with it.
|
||||
slot := emits % RECORD_STAGING;
|
||||
if staged[slot].pending record_read_back(r, slot, staged[slot]);
|
||||
ID3D11DeviceContext_CopyResource(s.device_context, s.staging[slot], s.latest);
|
||||
staged[slot] = .{pending = true, ticks = now, pointer = s.pointer_position, pointer_visible = s.pointer_visible};
|
||||
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
|
||||
ID3D11DeviceContext_CopyResource(gpu.device_context, gpu.staging[slot], gpu.latest);
|
||||
staged[slot] = .{pending = true, ticks = now};
|
||||
if !r.using_wgc staged[slot].pointer, staged[slot].pointer_visible = r.dxgi.pointer_position, r.dxgi.pointer_visible;
|
||||
emits += 1;
|
||||
last_emit = now;
|
||||
changed = false;
|
||||
@ -278,16 +374,16 @@ record_capture_proc :: (thread: *Thread) -> s64 {
|
||||
// What is still staged, oldest first.
|
||||
for 0..RECORD_STAGING - 1 {
|
||||
slot := (emits + it) % RECORD_STAGING;
|
||||
if staged[slot].pending record_read_back(r, slot, staged[slot]);
|
||||
if staged[slot].pending record_read_back(r, gpu, slot, staged[slot]);
|
||||
}
|
||||
atomic_swap(*r.capture_ended, true);
|
||||
signal(*r.ready);
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Copies a staging texture into a free frame, draws the cursor and queues it.
|
||||
record_read_back :: (r: *Recorder, slot: s64, staged: Staged) {
|
||||
s := *r.source;
|
||||
// Copies a staging texture into a free frame, draws the cursor (Desktop Duplication leaves it out)
|
||||
// and queues it.
|
||||
record_read_back :: (r: *Recorder, gpu: *Capture_Gpu, slot: s64, staged: Staged) {
|
||||
t := ticks_now();
|
||||
lock(*r.mutex);
|
||||
if !r.free_count { // every frame is waiting: drop the oldest
|
||||
@ -303,7 +399,7 @@ record_read_back :: (r: *Recorder, slot: s64, staged: Staged) {
|
||||
frame := *r.frames[index];
|
||||
|
||||
mapped: D3D11_MAPPED_SUBRESOURCE;
|
||||
hr := ID3D11DeviceContext_Map(s.device_context, s.staging[slot], 0, .READ, 0, *mapped);
|
||||
hr := ID3D11DeviceContext_Map(gpu.device_context, gpu.staging[slot], 0, .READ, 0, *mapped);
|
||||
if FAILED(hr) {
|
||||
lock(*r.mutex); r.free[r.free_count] = index; r.free_count += 1; unlock(*r.mutex);
|
||||
record_fail(r, hresult_text("reading the frame back", hr));
|
||||
@ -311,10 +407,10 @@ record_read_back :: (r: *Recorder, slot: s64, staged: Staged) {
|
||||
}
|
||||
row := cast(s64) r.width * 4;
|
||||
for y: 0..r.height - 1 memcpy(frame.pixels.data + y * row, cast(*u8) mapped.pData + cast(s64) y * mapped.RowPitch, row);
|
||||
ID3D11DeviceContext_Unmap(s.device_context, s.staging[slot], 0);
|
||||
ID3D11DeviceContext_Unmap(gpu.device_context, gpu.staging[slot], 0);
|
||||
if r.options.cursor && staged.pointer_visible {
|
||||
cursor_draw(frame.pixels.data, r.width, r.height, *s.shape,
|
||||
staged.pointer.x - cast(s32) s.box.left, staged.pointer.y - cast(s32) s.box.top);
|
||||
cursor_draw(frame.pixels.data, r.width, r.height, *r.dxgi.shape,
|
||||
staged.pointer.x - cast(s32) gpu.box.left, staged.pointer.y - cast(s32) gpu.box.top);
|
||||
}
|
||||
frame.ticks = staged.ticks;
|
||||
|
||||
@ -383,19 +479,15 @@ record_fail :: (r: *Recorder, error: string) {
|
||||
atomic_swap(*r.stopping, true);
|
||||
}
|
||||
|
||||
ticks_now :: () -> s64 {
|
||||
t: LARGE_INTEGER;
|
||||
QueryPerformanceCounter(*t);
|
||||
return t.QuadPart;
|
||||
}
|
||||
|
||||
rect_inside :: (inner: RECT, outer: RECT) -> bool {
|
||||
return inner.left >= outer.left && inner.top >= outer.top && inner.right <= outer.right && inner.bottom <= outer.bottom;
|
||||
}
|
||||
|
||||
#scope_export
|
||||
|
||||
#load "capture_gpu.jai";
|
||||
#load "dxgi_source.jai";
|
||||
#load "wgc_source.jai";
|
||||
#load "encoder.jai";
|
||||
#load "cursor.jai";
|
||||
|
||||
|
||||
406
modules/Screen_Record/wgc_source.jai
Normal file
406
modules/Screen_Record/wgc_source.jai
Normal file
@ -0,0 +1,406 @@
|
||||
// Windows.Graphics.Capture: the screen as Windows' own capture API hands it over, for when Desktop
|
||||
// Duplication cannot run (dxgi_source.jai). It works across GPUs, hands over frames the right way up
|
||||
// on a rotated screen, and draws the cursor in itself.
|
||||
//
|
||||
// It is WinRT, bound here by hand: the few interfaces it needs, as vtables after IInspectable's six
|
||||
// methods. Everything is made and used on the capture thread, which is in the multithreaded apartment
|
||||
// (RoInitialize), with a free-threaded frame pool, so no dispatcher queue or message loop is needed.
|
||||
//
|
||||
// A frame's arrival sets an event, through a small handler object (Frame_Handler); if Windows will not
|
||||
// take the handler, the pool is polled every millisecond instead. Windows 11 lets the yellow border it
|
||||
// draws around a screen being captured be turned off (IsBorderRequired); Windows 10 always draws it.
|
||||
|
||||
Wgc_Source :: struct {
|
||||
#as using gpu: Capture_Gpu;
|
||||
screen_size: Size_Int32; // the frames' size, as the pool was made for
|
||||
item: *Capture_Item;
|
||||
winrt_device: *IInspectable; // IDirect3DDevice, over gpu.device
|
||||
pool: *Frame_Pool;
|
||||
session: *Capture_Session;
|
||||
handler: Frame_Handler;
|
||||
handler_token: s64;
|
||||
polling: bool; // Windows would not take the handler
|
||||
border: string; // what came of turning the border off, for the log
|
||||
}
|
||||
|
||||
// Starts capturing the screen. region is in the screen's pixels and lies on it.
|
||||
wgc_open :: (s: *Wgc_Source, screen: Record_Screen, region: RECT, cursor: bool, testing: Record_Testing = 0) -> ok: bool, error: string {
|
||||
ok, error := gpu_open(s, screen, region);
|
||||
if !ok return false, error;
|
||||
|
||||
session_statics: *Session_Statics;
|
||||
hr := activation_factory("Windows.Graphics.Capture.GraphicsCaptureSession", IID_SESSION_STATICS, xx *session_statics);
|
||||
if FAILED(hr) return false, hresult_text("Windows.Graphics.Capture is not on this system", hr);
|
||||
supported: u8;
|
||||
session_statics.vtable.IsSupported(session_statics, *supported);
|
||||
com_release(*session_statics);
|
||||
if !supported return false, "Windows.Graphics.Capture is not supported here";
|
||||
|
||||
interop: *Capture_Item_Interop;
|
||||
hr = activation_factory("Windows.Graphics.Capture.GraphicsCaptureItem", IID_ITEM_INTEROP, xx *interop);
|
||||
if FAILED(hr) return false, hresult_text("IGraphicsCaptureItemInterop", hr);
|
||||
hr = interop.vtable.CreateForMonitor(interop, screen.monitor, *IID_ITEM, xx *s.item);
|
||||
com_release(*interop);
|
||||
if FAILED(hr) return false, hresult_text("a capture item for the screen", hr);
|
||||
s.item.vtable.get_Size(s.item, *s.screen_size);
|
||||
desktop_width := screen.desktop.right - screen.desktop.left;
|
||||
desktop_height := screen.desktop.bottom - screen.desktop.top;
|
||||
if s.screen_size.width != desktop_width || s.screen_size.height != desktop_height {
|
||||
return false, tprint("the screen is % x %, but this process sees it as % x %: it must be aware of each monitor's DPI",
|
||||
s.screen_size.width, s.screen_size.height, desktop_width, desktop_height);
|
||||
}
|
||||
|
||||
dxgi_device: *IDXGIDevice;
|
||||
hr = IUnknown_QueryInterface(s.device, *uid(IDXGIDevice_UUID), xx *dxgi_device);
|
||||
if FAILED(hr) return false, hresult_text("the device's IDXGIDevice", hr);
|
||||
inspectable: *IInspectable;
|
||||
hr = CreateDirect3D11DeviceFromDXGIDevice(dxgi_device, *inspectable);
|
||||
com_release(*dxgi_device);
|
||||
if FAILED(hr) return false, hresult_text("CreateDirect3D11DeviceFromDXGIDevice", hr);
|
||||
hr = inspectable.vtable.QueryInterface(inspectable, *IID_DIRECT3D_DEVICE, xx *s.winrt_device);
|
||||
com_release(*inspectable);
|
||||
if FAILED(hr) return false, hresult_text("the device as IDirect3DDevice", hr);
|
||||
|
||||
pool_statics: *Frame_Pool_Statics2;
|
||||
hr = activation_factory("Windows.Graphics.Capture.Direct3D11CaptureFramePool", IID_POOL_STATICS2, xx *pool_statics);
|
||||
if FAILED(hr) return false, hresult_text("Direct3D11CaptureFramePool's statics", hr);
|
||||
hr = pool_statics.vtable.CreateFreeThreaded(pool_statics, s.winrt_device, PIXEL_FORMAT_B8G8R8A8, POOL_BUFFERS, size_packed(s.screen_size), *s.pool);
|
||||
com_release(*pool_statics);
|
||||
if FAILED(hr) return false, hresult_text("a free-threaded frame pool", hr);
|
||||
hr = s.pool.vtable.CreateCaptureSession(s.pool, s.item, *s.session);
|
||||
if FAILED(hr) return false, hresult_text("a capture session", hr);
|
||||
|
||||
// The cursor (Windows 10 2004 on) and the border (Windows 11): optional, by what the system has.
|
||||
session2: *Capture_Session2;
|
||||
if SUCCEEDED(s.session.vtable.QueryInterface(s.session, *IID_SESSION2, xx *session2)) {
|
||||
session2.vtable.put_IsCursorCaptureEnabled(session2, xx cursor);
|
||||
com_release(*session2);
|
||||
}
|
||||
session3: *Capture_Session3;
|
||||
if testing & .KEEP_BORDER {
|
||||
s.border = "border left on";
|
||||
} else if SUCCEEDED(s.session.vtable.QueryInterface(s.session, *IID_SESSION3, xx *session3)) {
|
||||
hr = session3.vtable.put_IsBorderRequired(session3, 0);
|
||||
s.border = ifx SUCCEEDED(hr) then "border off" else hresult_text("the border could not be turned off", hr);
|
||||
com_release(*session3);
|
||||
} else {
|
||||
s.border = "border always drawn (no IGraphicsCaptureSession3: Windows 10)";
|
||||
}
|
||||
|
||||
s.handler.vtable = *frame_handler_vtable;
|
||||
s.handler.event = CreateEventW(null, 0, 0, null);
|
||||
hr = ifx testing & .WGC_POLL then cast,no_check(HRESULT) 0x80004002 else s.pool.vtable.add_FrameArrived(s.pool, *s.handler, *s.handler_token);
|
||||
if FAILED(hr) {
|
||||
s.polling = true;
|
||||
timeBeginPeriod(1); // so polling every millisecond is every millisecond
|
||||
log("[record] FrameArrived would not take the handler (%); polling the frame pool.", hresult_text("add_FrameArrived", hr));
|
||||
}
|
||||
|
||||
hr = s.session.vtable.StartCapture(s.session);
|
||||
if FAILED(hr) return false, hresult_text("StartCapture", hr);
|
||||
return true, "";
|
||||
}
|
||||
|
||||
// Waits up to timeout_ms for a new frame; the newest's region goes into s.latest.
|
||||
wgc_next :: (s: *Wgc_Source, timeout_ms: u32) -> Capture_Next, error: string {
|
||||
frame := wgc_take_newest(s);
|
||||
if !frame {
|
||||
if s.polling {
|
||||
start := ticks_now();
|
||||
frequency: LARGE_INTEGER;
|
||||
QueryPerformanceFrequency(*frequency);
|
||||
while !frame && (ticks_now() - start) * 1000 / frequency.QuadPart < timeout_ms {
|
||||
sleep_milliseconds(1);
|
||||
frame = wgc_take_newest(s);
|
||||
}
|
||||
} else {
|
||||
WaitForSingleObject(s.handler.event, timeout_ms);
|
||||
frame = wgc_take_newest(s);
|
||||
}
|
||||
}
|
||||
if !frame return .NONE, "";
|
||||
defer close_and_release(frame);
|
||||
|
||||
// The screen changed size (a new resolution): the pool is made again; the region must still fit.
|
||||
size: Size_Int32;
|
||||
frame.vtable.get_ContentSize(frame, *size);
|
||||
if size.width != s.screen_size.width || size.height != s.screen_size.height {
|
||||
s.screen_size = size;
|
||||
hr := s.pool.vtable.Recreate(s.pool, s.winrt_device, PIXEL_FORMAT_B8G8R8A8, POOL_BUFFERS, size_packed(size));
|
||||
if FAILED(hr) return .FAILED, hresult_text("the frame pool could not be made again", hr);
|
||||
if cast(s32) s.box.right > size.width || cast(s32) s.box.bottom > size.height
|
||||
return .FAILED, tprint("the screen is now % x %, and the region is not on it", size.width, size.height);
|
||||
return .NONE, "";
|
||||
}
|
||||
|
||||
surface: *IInspectable;
|
||||
hr := frame.vtable.get_Surface(frame, *surface);
|
||||
if FAILED(hr) return .FAILED, hresult_text("the frame's surface", hr);
|
||||
defer com_release(*surface);
|
||||
access: *Dxgi_Interface_Access;
|
||||
hr = surface.vtable.QueryInterface(surface, *IID_DXGI_INTERFACE_ACCESS, xx *access);
|
||||
if FAILED(hr) return .FAILED, hresult_text("the surface's IDirect3DDxgiInterfaceAccess", hr);
|
||||
defer com_release(*access);
|
||||
texture: *ID3D11Texture2D;
|
||||
hr = access.vtable.GetInterface(access, *uid(ID3D11Texture2D_UUID), xx *texture);
|
||||
if FAILED(hr) return .FAILED, hresult_text("the surface's texture", hr);
|
||||
gpu_copy_region(s, texture);
|
||||
IUnknown_Release(texture);
|
||||
return .IMAGE, "";
|
||||
}
|
||||
|
||||
wgc_close :: (s: *Wgc_Source) {
|
||||
if s.pool && s.handler_token s.pool.vtable.remove_FrameArrived(s.pool, s.handler_token);
|
||||
if s.session close_and_release(s.session);
|
||||
if s.pool close_and_release(s.pool);
|
||||
s.session, s.pool = null, null;
|
||||
com_release(*s.item);
|
||||
com_release(*s.winrt_device);
|
||||
if s.handler.event CloseHandle(s.handler.event);
|
||||
if s.polling timeEndPeriod(1);
|
||||
gpu_close(s);
|
||||
s.* = .{};
|
||||
}
|
||||
|
||||
// The thread's apartment: WinRT on this thread from here on.
|
||||
wgc_thread_start :: () -> ok: bool {
|
||||
if !winrt_load() return false;
|
||||
return SUCCEEDED(winrt.RoInitialize(RO_INIT_MULTITHREADED));
|
||||
}
|
||||
|
||||
wgc_thread_end :: () {
|
||||
if winrt.RoUninitialize winrt.RoUninitialize();
|
||||
}
|
||||
|
||||
#scope_file
|
||||
|
||||
POOL_BUFFERS :: 2;
|
||||
PIXEL_FORMAT_B8G8R8A8 :: 87; // DirectXPixelFormat.B8G8R8A8UIntNormalized
|
||||
RO_INIT_MULTITHREADED :: 1;
|
||||
|
||||
// The pool's newest frame, the older ones let go; null when there is none.
|
||||
wgc_take_newest :: (s: *Wgc_Source) -> *Capture_Frame {
|
||||
newest: *Capture_Frame;
|
||||
while true {
|
||||
frame: *Capture_Frame;
|
||||
if FAILED(s.pool.vtable.TryGetNextFrame(s.pool, *frame)) || !frame break;
|
||||
if newest close_and_release(newest);
|
||||
newest = frame;
|
||||
}
|
||||
return newest;
|
||||
}
|
||||
|
||||
// IClosable.Close, which hands a frame back to the pool at once, then Release.
|
||||
close_and_release :: (object: *$T) {
|
||||
closable: *Closable;
|
||||
if SUCCEEDED(object.vtable.QueryInterface(object, *IID_CLOSABLE, xx *closable)) {
|
||||
closable.vtable.Close(closable);
|
||||
closable.vtable.Release(closable);
|
||||
}
|
||||
object.vtable.Release(object);
|
||||
}
|
||||
|
||||
activation_factory :: (class_name: string, iid: GUID, factory: **void) -> HRESULT {
|
||||
wide := utf8_to_wide(class_name,, temp);
|
||||
length: u32;
|
||||
while wide[length] length += 1;
|
||||
name: HSTRING;
|
||||
if !winrt.loaded return cast,no_check(HRESULT) 0x80004001; // E_NOTIMPL: no WinRT on this system
|
||||
hr := winrt.WindowsCreateString(wide, length, *name);
|
||||
if FAILED(hr) return hr;
|
||||
defer winrt.WindowsDeleteString(name);
|
||||
iid_copy := iid;
|
||||
return winrt.RoGetActivationFactory(name, *iid_copy, factory);
|
||||
}
|
||||
|
||||
size_packed :: (size: Size_Int32) -> u64 { // SizeInt32 by value: 8 bytes, in one register
|
||||
width := cast(u64) cast,no_check(u32) size.width;
|
||||
height := cast(u64) cast,no_check(u32) size.height;
|
||||
return width | (height << 32);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- the FrameArrived handler
|
||||
|
||||
// TypedEventHandler<Direct3D11CaptureFramePool, Object>: a COM object of our own whose Invoke sets an
|
||||
// event. It lives in the Wgc_Source and is never freed by its count; it is unregistered first.
|
||||
Frame_Handler :: struct {
|
||||
vtable: *Frame_Handler_VTable;
|
||||
event: HANDLE;
|
||||
}
|
||||
|
||||
Frame_Handler_VTable :: struct {
|
||||
QueryInterface: (this: *Frame_Handler, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
AddRef: (this: *Frame_Handler) -> u32 #c_call;
|
||||
Release: (this: *Frame_Handler) -> u32 #c_call;
|
||||
Invoke: (this: *Frame_Handler, sender: *void, args: *void) -> HRESULT #c_call;
|
||||
}
|
||||
|
||||
frame_handler_vtable := Frame_Handler_VTable.{handler_query, handler_add_ref, handler_release, handler_invoke};
|
||||
|
||||
handler_query :: (this: *Frame_Handler, iid: *GUID, out: **void) -> HRESULT #c_call {
|
||||
if guid_equal(iid, *IID_UNKNOWN) || guid_equal(iid, *IID_AGILE_OBJECT) || guid_equal(iid, *IID_FRAME_ARRIVED_HANDLER) {
|
||||
out.* = this;
|
||||
return 0;
|
||||
}
|
||||
out.* = null;
|
||||
return cast,no_check(HRESULT) 0x80004002; // E_NOINTERFACE
|
||||
}
|
||||
|
||||
handler_add_ref :: (this: *Frame_Handler) -> u32 #c_call { return 1; }
|
||||
handler_release :: (this: *Frame_Handler) -> u32 #c_call { return 1; }
|
||||
|
||||
handler_invoke :: (this: *Frame_Handler, sender: *void, args: *void) -> HRESULT #c_call {
|
||||
SetEvent(this.event);
|
||||
return 0;
|
||||
}
|
||||
|
||||
guid_equal :: (a: *GUID, b: *GUID) -> bool #c_call {
|
||||
if a.Data1 != b.Data1 || a.Data2 != b.Data2 || a.Data3 != b.Data3 return false;
|
||||
for 0..7 if a.Data4[it] != b.Data4[it] return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- bindings
|
||||
|
||||
d3d11_dll :: #library,system "d3d11";
|
||||
kernel32_dll :: #library,system "kernel32";
|
||||
|
||||
HSTRING :: *void;
|
||||
|
||||
// WinRT's own calls, from combase.dll at run time: its import library is runtimeobject.lib, which
|
||||
// names no DLL of its own for #library to find.
|
||||
winrt: struct {
|
||||
loaded: bool;
|
||||
RoInitialize: (init_type: s32) -> HRESULT #c_call;
|
||||
RoUninitialize: () #c_call;
|
||||
RoGetActivationFactory: (class_id: HSTRING, iid: *GUID, factory: **void) -> HRESULT #c_call;
|
||||
WindowsCreateString: (source: *u16, length: u32, out: *HSTRING) -> HRESULT #c_call;
|
||||
WindowsDeleteString: (text: HSTRING) -> HRESULT #c_call;
|
||||
}
|
||||
|
||||
winrt_load :: () -> bool {
|
||||
if winrt.loaded return true;
|
||||
library := LoadLibraryW(utf8_to_wide("combase.dll",, temp));
|
||||
if !library return false;
|
||||
winrt.RoInitialize = xx GetProcAddress(library, "RoInitialize");
|
||||
winrt.RoUninitialize = xx GetProcAddress(library, "RoUninitialize");
|
||||
winrt.RoGetActivationFactory = xx GetProcAddress(library, "RoGetActivationFactory");
|
||||
winrt.WindowsCreateString = xx GetProcAddress(library, "WindowsCreateString");
|
||||
winrt.WindowsDeleteString = xx GetProcAddress(library, "WindowsDeleteString");
|
||||
winrt.loaded = winrt.RoInitialize && winrt.RoUninitialize && winrt.RoGetActivationFactory && winrt.WindowsCreateString && winrt.WindowsDeleteString;
|
||||
return winrt.loaded;
|
||||
}
|
||||
CreateDirect3D11DeviceFromDXGIDevice :: (device: *IDXGIDevice, out: **IInspectable) -> HRESULT #foreign d3d11_dll;
|
||||
SetEvent :: (event: HANDLE) -> s32 #foreign kernel32_dll;
|
||||
|
||||
Size_Int32 :: struct { width, height: s32; }
|
||||
|
||||
IID_UNKNOWN := #run guid("00000000-0000-0000-C000-000000000046");
|
||||
IID_AGILE_OBJECT := #run guid("94ea2b94-e9cc-49e0-c0ff-ee64ca8f5b90");
|
||||
IID_FRAME_ARRIVED_HANDLER := #run guid("51a947f7-79cf-5a3e-a3a5-1289cfa6dfe8"); // TypedEventHandler<Direct3D11CaptureFramePool, Object>
|
||||
IID_ITEM_INTEROP := #run guid("3628E81B-3CAC-4C60-B7F4-23CE0E0C3356");
|
||||
IID_ITEM := #run guid("79C3F95B-31F7-4EC2-A464-632EF5D30760");
|
||||
IID_SESSION_STATICS := #run guid("2224A540-5974-49AA-B232-0882536F4CB5");
|
||||
IID_SESSION2 := #run guid("2C39AE40-7D2E-5044-804E-8B6799D4CF9E");
|
||||
IID_SESSION3 := #run guid("F2CDD966-22AE-5EA1-9596-3A289344C3BE");
|
||||
IID_POOL_STATICS2 := #run guid("589B103F-6BBC-5DF5-A991-02E28B3B66D5");
|
||||
IID_DIRECT3D_DEVICE := #run guid("A37624AB-8D5F-4650-9D3E-9EAE3D9BC670");
|
||||
IID_DXGI_INTERFACE_ACCESS := #run guid("A9B3D012-3DF2-4EE3-B8D1-8695F457D3C1");
|
||||
IID_CLOSABLE := #run guid("30D5A829-7FA4-4026-83BB-D75BAE4EA99E");
|
||||
|
||||
guid :: (text: string) -> GUID {
|
||||
hex :: (s: string) -> u64 {
|
||||
value: u64;
|
||||
for 0..s.count - 1 {
|
||||
c := s[it];
|
||||
digit: u64 = ifx c >= #char "a" then cast(u64) (c - #char "a" + 10) else ifx c >= #char "A" then cast(u64) (c - #char "A" + 10) else cast(u64) (c - #char "0");
|
||||
value = value * 16 + digit;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
g: GUID;
|
||||
g.Data1 = cast(u32) hex(slice(text, 0, 8));
|
||||
g.Data2 = cast(u16) hex(slice(text, 9, 4));
|
||||
g.Data3 = cast(u16) hex(slice(text, 14, 4));
|
||||
for 0..1 g.Data4[it] = cast(u8) hex(slice(text, 19 + it * 2, 2));
|
||||
for 0..5 g.Data4[it + 2] = cast(u8) hex(slice(text, 24 + it * 2, 2));
|
||||
return g;
|
||||
}
|
||||
|
||||
// IUnknown, then IInspectable's three: every WinRT interface starts so.
|
||||
Inspectable_Methods :: struct {
|
||||
QueryInterface: (this: *void, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
AddRef: (this: *void) -> u32 #c_call;
|
||||
Release: (this: *void) -> u32 #c_call;
|
||||
GetIids: *void;
|
||||
GetRuntimeClassName: *void;
|
||||
GetTrustLevel: *void;
|
||||
}
|
||||
|
||||
IInspectable :: struct { vtable: *Inspectable_Methods; }
|
||||
|
||||
Closable :: struct { vtable: *struct { using base: Inspectable_Methods; Close: (this: *Closable) -> HRESULT #c_call; }; }
|
||||
|
||||
Session_Statics :: struct { vtable: *struct { using base: Inspectable_Methods; IsSupported: (this: *Session_Statics, out: *u8) -> HRESULT #c_call; }; }
|
||||
|
||||
Capture_Item_Interop :: struct { vtable: *struct {
|
||||
QueryInterface: (this: *void, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
AddRef: (this: *void) -> u32 #c_call;
|
||||
Release: (this: *void) -> u32 #c_call;
|
||||
CreateForWindow: (this: *Capture_Item_Interop, window: HWND, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
CreateForMonitor: (this: *Capture_Item_Interop, monitor: HMONITOR, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Capture_Item :: struct { vtable: *struct {
|
||||
using base: Inspectable_Methods;
|
||||
get_DisplayName: *void;
|
||||
get_Size: (this: *Capture_Item, out: *Size_Int32) -> HRESULT #c_call;
|
||||
add_Closed: *void;
|
||||
remove_Closed: *void;
|
||||
}; }
|
||||
|
||||
Frame_Pool_Statics2 :: struct { vtable: *struct {
|
||||
using base: Inspectable_Methods;
|
||||
CreateFreeThreaded: (this: *Frame_Pool_Statics2, device: *IInspectable, format: s32, buffers: s32, size: u64, out: **Frame_Pool) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Frame_Pool :: struct { vtable: *struct {
|
||||
using base: Inspectable_Methods;
|
||||
Recreate: (this: *Frame_Pool, device: *IInspectable, format: s32, buffers: s32, size: u64) -> HRESULT #c_call;
|
||||
TryGetNextFrame: (this: *Frame_Pool, out: **Capture_Frame) -> HRESULT #c_call;
|
||||
add_FrameArrived: (this: *Frame_Pool, handler: *Frame_Handler, token: *s64) -> HRESULT #c_call;
|
||||
remove_FrameArrived: (this: *Frame_Pool, token: s64) -> HRESULT #c_call;
|
||||
CreateCaptureSession: (this: *Frame_Pool, item: *Capture_Item, out: **Capture_Session) -> HRESULT #c_call;
|
||||
get_DispatcherQueue: *void;
|
||||
}; }
|
||||
|
||||
Capture_Frame :: struct { vtable: *struct {
|
||||
using base: Inspectable_Methods;
|
||||
get_Surface: (this: *Capture_Frame, out: **IInspectable) -> HRESULT #c_call;
|
||||
get_SystemRelativeTime: (this: *Capture_Frame, out: *s64) -> HRESULT #c_call;
|
||||
get_ContentSize: (this: *Capture_Frame, out: *Size_Int32) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Capture_Session :: struct { vtable: *struct {
|
||||
using base: Inspectable_Methods;
|
||||
StartCapture: (this: *Capture_Session) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Capture_Session2 :: struct { vtable: *struct {
|
||||
using base: Inspectable_Methods;
|
||||
get_IsCursorCaptureEnabled: (this: *Capture_Session2, out: *u8) -> HRESULT #c_call;
|
||||
put_IsCursorCaptureEnabled: (this: *Capture_Session2, value: u8) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Capture_Session3 :: struct { vtable: *struct {
|
||||
using base: Inspectable_Methods;
|
||||
get_IsBorderRequired: (this: *Capture_Session3, out: *u8) -> HRESULT #c_call;
|
||||
put_IsBorderRequired: (this: *Capture_Session3, value: u8) -> HRESULT #c_call;
|
||||
}; }
|
||||
|
||||
Dxgi_Interface_Access :: struct { vtable: *struct {
|
||||
QueryInterface: (this: *void, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
AddRef: (this: *void) -> u32 #c_call;
|
||||
Release: (this: *void) -> u32 #c_call;
|
||||
GetInterface: (this: *Dxgi_Interface_Access, iid: *GUID, out: **void) -> HRESULT #c_call;
|
||||
}; }
|
||||
@ -3,6 +3,14 @@
|
||||
// there are and on which adapter, which duplication call and encoder worked, frames captured, written
|
||||
// and dropped, time per stage, and the file's size. It also answers what the plan leaves to phase 1:
|
||||
//
|
||||
// -capture HOW auto (Desktop Duplication, else Windows.Graphics.Capture), dxgi or wgc.
|
||||
// -border-test counts yellow pixels along the screen's edges, with GDI, before recording and a
|
||||
// second in: does Windows draw its capture border (Windows.Graphics.Capture)?
|
||||
// -keep-border leaves Windows.Graphics.Capture's border on: the border test's control.
|
||||
// -simulate-unsupported Desktop Duplication fails as on a laptop with two GPUs: the fallback's path.
|
||||
// -poll Windows.Graphics.Capture polled, as when it will not take its frame handler.
|
||||
// -cursor-test puts the mouse in the middle of the region while recording, and back after;
|
||||
// with -snapshot, cursor.png is the frame around it, four times as large.
|
||||
// -legacy the plain DuplicateOutput, not DuplicateOutput1: does it fail for a process aware
|
||||
// of each monitor's DPI, as mcap is (this probe is, unless -dpi-unaware)?
|
||||
// -cuda makes a CUDA context first, as onnxruntime does in mcap: does duplication still work?
|
||||
@ -18,7 +26,8 @@
|
||||
// Build: jai first.jai - record_probe
|
||||
// Run: bin/record-probe [-list] [-screen N] [-region X Y W H] [-seconds S] [-fps F] [-encoder NAME]
|
||||
// [-out FILE.mp4] [-no-cursor] [-legacy] [-cuda] [-exclude-test] [-snapshot DIR]
|
||||
// [-animate] [-dpi-unaware]
|
||||
// [-animate] [-capture auto|dxgi|wgc] [-border-test] [-keep-border] [-cursor-test]
|
||||
// [-simulate-unsupported] [-poll] [-dpi-unaware]
|
||||
// Exit code: 0 recorded, 1 failed, 3 Desktop Duplication cannot run here (the fallback's case).
|
||||
|
||||
main :: () {
|
||||
@ -26,7 +35,7 @@ main :: () {
|
||||
options: Record_Options;
|
||||
options.path = tprint("%/record-probe.mp4", get_working_directory());
|
||||
seconds: float64 = 10;
|
||||
list_only, cuda, exclude_test, dpi_unaware, animate := false, false, false, false, false;
|
||||
list_only, cuda, exclude_test, dpi_unaware, animate, border_test, cursor_test := false, false, false, false, false, false, false;
|
||||
snapshot_folder: string;
|
||||
i := 1;
|
||||
number :: (index: s64) -> s32 #expand { return cast(s32) string_to_int(`args[index]); }
|
||||
@ -51,6 +60,19 @@ main :: () {
|
||||
if a == "-exclude-test" { exclude_test = true; i += 1; continue; }
|
||||
if a == "-dpi-unaware" { dpi_unaware = true; i += 1; continue; }
|
||||
if a == "-animate" { animate = true; i += 1; continue; }
|
||||
if a == "-border-test" { border_test = true; i += 1; continue; }
|
||||
if a == "-keep-border" { options.testing |= .KEEP_BORDER; i += 1; continue; }
|
||||
if a == "-simulate-unsupported" { options.testing |= .DXGI_UNSUPPORTED; i += 1; continue; }
|
||||
if a == "-poll" { options.testing |= .WGC_POLL; i += 1; continue; }
|
||||
if a == "-cursor-test" { cursor_test = true; i += 1; continue; }
|
||||
if a == "-capture" && more >= 1 {
|
||||
how := args[i + 1];
|
||||
if how == "auto" options.capture = .AUTO;
|
||||
else if how == "dxgi" options.capture = .DXGI;
|
||||
else if how == "wgc" options.capture = .WGC;
|
||||
else { print("-capture takes auto, dxgi or wgc\n"); exit(2); }
|
||||
i += 2; continue;
|
||||
}
|
||||
if a == "-check" && more >= 1 {
|
||||
image, info, ok, error := video_poster(args[i + 1]);
|
||||
if !ok { print("%: cannot be read: %\n", args[i + 1], error); exit(1); }
|
||||
@ -63,7 +85,13 @@ main :: () {
|
||||
|
||||
if !dpi_unaware {
|
||||
DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2 :: cast(HANDLE) -4;
|
||||
if !SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) print("Could not become per-monitor DPI aware (v2).\n");
|
||||
if !SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) {
|
||||
error := GetLastError();
|
||||
ERROR_ACCESS_DENIED :: 5;
|
||||
if error != ERROR_ACCESS_DENIED print("Could not become per-monitor DPI aware (v2): error %\n", error);
|
||||
// Access denied: the awareness was set already, as it can be once only. dxgi_open and wgc_open
|
||||
// refuse when the desktop they are told of is scaled.
|
||||
}
|
||||
}
|
||||
print("DPI awareness: %\n", ifx dpi_unaware then "none" else "per monitor (v2), as mcap");
|
||||
|
||||
@ -83,6 +111,10 @@ main :: () {
|
||||
if !ok exit(1);
|
||||
}
|
||||
|
||||
border_screen := screens[ifx options.screen >= 0 then options.screen else 0].desktop;
|
||||
border_before: s64;
|
||||
if border_test border_before = yellow_edge_pixels(border_screen);
|
||||
|
||||
recorder: Recorder;
|
||||
ok, error, fall_back := record_start(*recorder, options);
|
||||
if !ok {
|
||||
@ -91,17 +123,26 @@ main :: () {
|
||||
exit(ifx fall_back then cast(s32) 3 else 1);
|
||||
}
|
||||
stats := record_stats(*recorder);
|
||||
if stats.fell_back_from print("Desktop Duplication could not run (%), so:\n", stats.fell_back_from);
|
||||
print("Recording % x % for % s at up to % fps with % and %, to %\n", stats.width, stats.height, seconds, options.fps, stats.source, stats.encoder, options.path);
|
||||
|
||||
region := options.region;
|
||||
if region.right <= region.left region = screens[ifx options.screen >= 0 then options.screen else 0].desktop;
|
||||
excluded, control: HWND;
|
||||
if exclude_test excluded, control = exclusion_windows(region);
|
||||
saved_cursor: POINT;
|
||||
cursor_at: POINT;
|
||||
if cursor_test {
|
||||
GetCursorPos(*saved_cursor);
|
||||
cursor_at = .{region.left + (region.right - region.left) / 2, region.top + (region.bottom - region.top) / 2};
|
||||
SetCursorPos(cursor_at.x, cursor_at.y);
|
||||
}
|
||||
mover: HWND;
|
||||
if animate mover = square_window("record-probe animate", 0x0000A5FF, region.left, region.top, 120);
|
||||
|
||||
start := current_time_monotonic();
|
||||
last_print := start;
|
||||
border_during: s64 = -1;
|
||||
while to_float64_seconds(current_time_monotonic() - start) < seconds {
|
||||
pump_messages();
|
||||
if mover { // back and forth across the region, about 240 times a second
|
||||
@ -120,12 +161,15 @@ main :: () {
|
||||
print(" % s: % captured, % sent (% repeats), % written, % dropped, % lost, % KB\n",
|
||||
FF(to_float64_seconds(last_print - start), 0), s.captured, s.emitted, s.repeats, s.written, s.dropped, s.lost, s.bytes / 1024);
|
||||
}
|
||||
if border_test && border_during < 0 && to_float64_seconds(current_time_monotonic() - start) >= 1.5
|
||||
border_during = yellow_edge_pixels(border_screen);
|
||||
failed, _ := record_failed(*recorder);
|
||||
if failed break;
|
||||
}
|
||||
|
||||
last, width, height := record_last_frame(*recorder);
|
||||
if mover DestroyWindow(mover);
|
||||
if cursor_test SetCursorPos(saved_cursor.x, saved_cursor.y);
|
||||
stopped, stop_error := record_stop(*recorder);
|
||||
s := recorder.stats;
|
||||
print("\n%\n", ifx stopped then "Recorded." else tprint("Recording failed: %", stop_error));
|
||||
@ -159,9 +203,28 @@ main :: () {
|
||||
DestroyWindow(control);
|
||||
}
|
||||
|
||||
if border_test {
|
||||
print(" border: % yellow pixels along the screen's edges before recording, % while: %\n", border_before, border_during,
|
||||
ifx border_during < 0 then "not measured (too short)"
|
||||
else ifx border_during > border_before + 200 then "a capture border is drawn"
|
||||
else "no capture border");
|
||||
}
|
||||
|
||||
if snapshot_folder {
|
||||
make_directory_if_it_does_not_exist(snapshot_folder, recursive = true);
|
||||
rgba := bgra_to_rgba(last);
|
||||
if cursor_test { // 48 x 48 around where the mouse was put, four times as large
|
||||
SIDE :: 48; SCALE :: 4;
|
||||
crop := NewArray(SIDE * SCALE * SIDE * SCALE * 4, u8,, temp);
|
||||
cx, cy := cursor_at.x - region.left - 8, cursor_at.y - region.top - 8;
|
||||
for y: 0..SIDE * SCALE - 1 for x: 0..SIDE * SCALE - 1 {
|
||||
sx, sy := clamp(cx + x / SCALE, 0, width - 1), clamp(cy + y / SCALE, 0, height - 1);
|
||||
memcpy(crop.data + (y * SIDE * SCALE + x) * 4, rgba.data + (cast(s64) sy * width + sx) * 4, 4);
|
||||
}
|
||||
cursor_path := tprint("%/cursor.png", snapshot_folder);
|
||||
stbi_write_png(temp_c_string(cursor_path), SIDE * SCALE, SIDE * SCALE, 4, crop.data, SIDE * SCALE * 4);
|
||||
print(" cursor: %\n", cursor_path);
|
||||
}
|
||||
captured_path := tprint("%/captured.png", snapshot_folder);
|
||||
stbi_write_png(temp_c_string(captured_path), width, height, 4, rgba.data, width * 4);
|
||||
image, info, decoded, decode_error := video_poster(options.path);
|
||||
@ -210,6 +273,40 @@ square_window :: (class_name: string, colour: u32, x: s32, y: s32, side: s32) ->
|
||||
WS_POPUP | WS_VISIBLE, x, y, side, side, null, null, instance, null);
|
||||
}
|
||||
|
||||
// Pixels along a screen's outer 6 pixels that look like Windows' yellow capture border, with GDI.
|
||||
yellow_edge_pixels :: (screen: RECT) -> s64 {
|
||||
width, height := screen.right - screen.left, screen.bottom - screen.top;
|
||||
desktop := GetDC(null);
|
||||
defer ReleaseDC(null, desktop);
|
||||
dc := CreateCompatibleDC(desktop);
|
||||
defer DeleteDC(dc);
|
||||
bitmap := CreateCompatibleBitmap(desktop, width, height);
|
||||
defer DeleteObject(bitmap);
|
||||
old := SelectObject(dc, bitmap);
|
||||
BitBlt(dc, 0, 0, width, height, desktop, screen.left, screen.top, cast(ROP) (cast(u32) ROP.SRCCOPY | 0x40000000)); // CAPTUREBLT
|
||||
SelectObject(dc, old);
|
||||
|
||||
info: BITMAPINFO;
|
||||
info.bmiHeader.biSize = size_of(BITMAPINFOHEADER);
|
||||
info.bmiHeader.biWidth = width;
|
||||
info.bmiHeader.biHeight = -height;
|
||||
info.bmiHeader.biPlanes = 1;
|
||||
info.bmiHeader.biBitCount = 32;
|
||||
info.bmiHeader.biCompression = BI_RGB;
|
||||
pixels := NewArray(cast(s64) width * height * 4, u8,, temp);
|
||||
GetDIBits(dc, bitmap, 0, xx height, pixels.data, *info, DIB_RGB_COLORS);
|
||||
|
||||
EDGE :: 6;
|
||||
count: s64;
|
||||
for y: 0..height - 1 for x: 0..width - 1 {
|
||||
if x >= EDGE && x < width - EDGE && y >= EDGE && y < height - EDGE continue;
|
||||
p := pixels.data + (cast(s64) y * width + x) * 4;
|
||||
b, g, r := cast(s32) p[0], cast(s32) p[1], cast(s32) p[2];
|
||||
if r > 200 && g > 120 && b < 90 && r - b > 130 count += 1;
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
// A window's rectangle, in the region's pixels, a few pixels in from its edges.
|
||||
window_rect_in :: (window: HWND, region: RECT) -> RECT {
|
||||
r: RECT;
|
||||
@ -287,6 +384,7 @@ FF :: (value: float64, decimals: s64) -> FormatFloat #expand {
|
||||
|
||||
SetProcessDpiAwarenessContext :: (value: HANDLE) -> BOOL #foreign user32_dll;
|
||||
SetWindowDisplayAffinity :: (window: HWND, affinity: u32) -> BOOL #foreign user32_dll;
|
||||
SetCursorPos :: (x: s32, y: s32) -> BOOL #foreign user32_dll;
|
||||
user32_dll :: #library,system "user32";
|
||||
|
||||
#import "Basic";
|
||||
|
||||
@ -68,7 +68,7 @@
|
||||
|
||||
<header>
|
||||
<h1>Screen video capture in mcap</h1>
|
||||
<p>Planning document · 2026-10-10 · for mcap after 0.1.1 (still captures with PrintScreen) · <strong>phase 1 built, mcap 0.1.3</strong></p>
|
||||
<p>Planning document · 2026-10-10 · for mcap after 0.1.1 (still captures with PrintScreen) · <strong>phases 1 and 2 built, mcap 0.1.4</strong></p>
|
||||
</header>
|
||||
|
||||
<div class="summary">
|
||||
@ -81,6 +81,7 @@
|
||||
<li><span class="tag decided">Decided</span> <strong>A recording becomes an <code>.mcap</code></strong> in <em>Screen Captures</em>, the folder stills go to. It's stored the way imported videos are: the MP4 as the original, a frame from it for the thumbnail, OCR and labels.</li>
|
||||
<li>It's built in five phases. Phase 1 is a command-line probe that answers the two-GPU question on your laptop before anything depends on it.</li>
|
||||
<li><span class="tag decided">Done</span> <strong>Phase 1</strong> (2026-10-10, mcap 0.1.3): <code>modules/Screen_Record</code> and <code>record-probe</code>. 60 s of 4K at 30 fps with NVENC and no dropped frames. The two-GPU failure couldn't be reproduced, because this laptop's screens are currently driven by the NVIDIA GPU (see <a href="#phases">phase 1's result</a>).</li>
|
||||
<li><span class="tag decided">Done</span> <strong>Phase 2</strong> (2026-10-10, mcap 0.1.4): the Windows.Graphics.Capture fallback. Automatic mode falls back to it on a simulated <code>DXGI_ERROR_UNSUPPORTED</code>. It records 4K at 60 fps with nothing dropped, draws the cursor exactly as Desktop Duplication's code does, and the yellow border can be turned off on Windows 11.</li>
|
||||
<li><span class="tag later">Later</span> Audio is phase 5. Recording one window, and regions that span two screens, come after.</li>
|
||||
</ul>
|
||||
</div>
|
||||
@ -321,7 +322,7 @@ Captured_Frame :: struct {
|
||||
</div>
|
||||
|
||||
<div class="phase">
|
||||
<h3>Phase 2: Windows.Graphics.Capture fallback</h3>
|
||||
<h3>Phase 2: Windows.Graphics.Capture fallback <span class="tag decided">Done</span></h3>
|
||||
<p>The second source behind the same interface: WinRT bindings, polling <code>TryGetNextFrame</code>, the cursor and border settings. The fallback rules from 3.2, and <code>-capture dxgi|wgc|auto</code> on the probe.</p>
|
||||
<p><strong>Done when:</strong></p>
|
||||
<ul>
|
||||
@ -329,6 +330,35 @@ Captured_Frame :: struct {
|
||||
<li><code>auto</code> picks Desktop Duplication where it works and Windows.Graphics.Capture where it doesn't. The latter is shown either on the two-GPU laptop or on a rotated screen.</li>
|
||||
<li>Whether the yellow border can be turned off is recorded.</li>
|
||||
</ul>
|
||||
<div class="note"><strong>Result (2026-10-10).</strong> Release build, the same laptop and screens as phase 1.
|
||||
<ul>
|
||||
<li><strong>Built:</strong>
|
||||
<ul>
|
||||
<li><code>wgc_source.jai</code>: hand-written bindings for eight WinRT interfaces.</li>
|
||||
<li><code>capture_gpu.jai</code>: what both sources share, namely the device on the screen's adapter, the region copy and the staging ring. Each source is now only "the next image's region into a texture".</li>
|
||||
<li><code>Record_Options.capture</code>: <code>.AUTO</code>, <code>.DXGI</code> or <code>.WGC</code>.</li>
|
||||
<li>The probe's <code>-capture auto|dxgi|wgc</code>.</li>
|
||||
</ul></li>
|
||||
<li><strong>The fallback:</strong>
|
||||
<ul>
|
||||
<li><code>auto</code> picks Desktop Duplication where it works.</li>
|
||||
<li>With <code>-simulate-unsupported</code>, which makes duplication fail with <code>DXGI_ERROR_UNSUPPORTED</code> as on a two-GPU laptop, <code>auto</code> logs why and records with Windows.Graphics.Capture.</li>
|
||||
<li><code>-capture dxgi</code> with the same simulated failure refuses, with exit code 3.</li>
|
||||
<li>The real two-GPU case is shelved: this laptop's GPU is pinned in the BIOS.</li>
|
||||
</ul></li>
|
||||
<li><strong>Recording:</strong> 4K at 60 fps with motion: 56.6 frames a second taken and every one encoded, none dropped. Windows.Graphics.Capture delivers a little under the screen's 60 Hz; Desktop Duplication delivered 59.3. A 641 × 481 region on the internal screen gives the same frame as Desktop Duplication's in phase 1.</li>
|
||||
<li><strong>The border:</strong> <code>IsBorderRequired = false</code> works for mcap as an unpackaged app on Windows 11, with no access request. A GDI sample of the screen's outer 6 pixels found no yellow pixels with the border off, and 14,214 with it left on (<code>-keep-border</code>, the control).</li>
|
||||
<li><strong>The cursor:</strong> with the mouse put in the region (<code>-cursor-test</code>), Windows.Graphics.Capture's own cursor and the one <code>cursor.jai</code> draws into Desktop Duplication's frames are the same, pixel for pixel, by eye at 4×.</li>
|
||||
<li><strong><code>WDA_EXCLUDEFROMCAPTURE</code></strong> is respected by Windows.Graphics.Capture too: the excluded square is 0% present and the control square 100%.</li>
|
||||
<li><strong>Waiting for frames:</strong> each new frame sets an event, through a small COM handler for <code>FrameArrived</code>, which Windows took. The polling fallback, for when Windows refuses the handler (<code>-poll</code>), takes 41 frames a second of a 60 Hz screen, enough for a 30 fps recording (29.0 sent), at a 1 ms timer resolution.</li>
|
||||
</ul>
|
||||
<p><strong>Changes from the plan:</strong></p>
|
||||
<ul>
|
||||
<li><strong>Events, not polling.</strong> The plan said to poll <code>TryGetNextFrame</code>, to avoid writing a WinRT event handler in Jai. The handler took about 30 lines, so it's the default, and polling is the fallback.</li>
|
||||
<li><strong>The source opens on the capture thread</strong>, for both methods. WinRT objects belong to the apartment that made them, so <code>record_start</code> starts the capture thread and waits for it to say whether its source opened.</li>
|
||||
<li><strong>The WinRT calls</strong> (<code>RoInitialize</code>, <code>RoGetActivationFactory</code>, HSTRINGs) are loaded from <code>combase.dll</code> at run time. Their import library, <code>runtimeobject.lib</code>, names no DLL of its own, which Jai's <code>#library</code> needs.</li>
|
||||
<li><strong>Test switches:</strong> <code>Record_Options.testing</code> holds three, for the probe: simulate the two-GPU failure, force polling, keep the border.</li>
|
||||
</ul></div>
|
||||
</div>
|
||||
|
||||
<div class="phase">
|
||||
|
||||
Loading…
Reference in New Issue
Block a user