UIA provider isn't reliably detected on startup of example program

[mwcampbell]

Steps to reproduce:

1. Check out and build this commit of AccessKit.
2. On x64 Windows 10 build 19043.1237 (21H1), run NVDA 2021.2.
3. Run the hello_world example program under platforms/windows/examples.

Expected result: NVDA should say "Hello world window". This indicates that the UIA provider is properly exposed and NVDA is recognizing the window as a UIA window.

Observed: NVDA says "Hello world". This is what it says for a window for which it can't detect a UIA provider. However, if you reverse the order of the steps (start the program and then start NVDA), it says "Hello world window", indicating that it detected the UIA provider.

My analysis: I know that NVDA is behaving as it is because when it calls UiaHasServerSideProvider on the newly created window, that function is returning false. But I don't know why UiaHasServerSideProvider is returning false for that window. The window procedure handles WM_GETOBJECT by creating the UIA provider and passing that along with the WPARAM and LPARAM to UiaReturnRawElementProvider. It doesn't do anything else with the WPARAM and LPARAM, since I want to let UIA Core handle both UIA and MSAA requests. So I guess there's some kind of race condition where, immediately after the window is created, UIA Core running in NVDA can't connect to UIA Core running in the example program. FWIW, NVDA is 32-bit x86 while the example program is x64.

I have a workaround. If I call CoInitializeEx with COINIT_MULTITHREADED at the start of the program, then this unexpected behavior doesn't happen. But if I either omit a call to CoInitializeEx or call it with COINIT_APARTMENTTHREADED, then I get this problem. So for now, I can just use COINIT_MULTITHREADED, but since AccessKit is a library aiming for wide adoption, I want it to work in any of these cases if possible. Also, it doesn't seem to matter whether I use ProviderOptions_UseComThreading or not, though I'd prefer not to.

I've reported this to one of my former colleagues at Microsoft and his manager, since this looks like it might be a bug in UIA Core.

[DataTriny]

I can't reproduce the issue you describe (same Windows build, same Rust architecture, same NVDA version). Running the example from the commit linked above, I get the exact same result as with the next commit which introduce the workaround.
But just to be clear: the word "window" after "Hello world" should come from NVDA, hence in my case be translated to reflect the role of the element? If I change the name of the root node, NVDA still reads "Hello world" from CreateWindowExA, which is normal I guess since you haven't implemented IRawElementProviderSimple::GetPropertyValue(UIA_NamePropertyId, ...) yet.

For reference, here is what NVDA has to say about the window:

name: 'Hello world'
role: ROLE_WINDOW
roleText: None
states: STATE_FOCUSABLE, STATE_FOCUSED
isFocusable: True
hasFocus: True
Python object: <NVDAObjects.UIA.UIA object at 0x06D1FCB0>
Python class mro: (<class 'NVDAObjects.UIA.UIA'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'documentBase.TextContainerObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <class 'garbageHandler.TrackedObject'>, <class 'object'>)
description: ''
location: RectLTWH(left=26, top=26, width=1440, height=759)
value: None
appModule: <'appModuleHandler' (appName 'hello_world', process ID 6892) at address 2f537f0>
appModule.productName: exception: No version information
appModule.productVersion: exception: No version information
TextInfo: <class 'displayModel.DisplayModelTextInfo'>
windowHandle: 199218
windowClassName: 'window'
windowControlID: 0
windowStyle: 349110272
extendedWindowStyle: 256
windowThreadID: 7052
windowText: 'Hello world'
displayText: ''
UIAElement: <POINTER(IUIAutomationElement) ptr=0x38490f8 at 6ce1300>
UIA automationID: 
UIA frameworkID: Win32
UIA runtimeID: (42, 199218)
UIA providerDescription: [pid:19880,providerId:0x30A32 Main:Nested [pid:6892,providerId:0x30A32 Main(parent link):Unidentified Provider (unmanaged:hello_world.exe)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:UIAutomationCore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:UIAutomationCore.dll)]
UIA className: window
UIA patterns available: TransformPattern, WindowPattern, LegacyIAccessiblePattern

Hope it helps.

[mwcampbell]

@DataTriny Thanks for looking at this. If you get UIA stuff in the NVDA log, rather than IAccessible stuff, even in the situation I described where I had this problem, then the problem is indeed not happening for you.

Let's see what the team at Microsoft says.

[DataTriny]

Because of the potential race condition you mentioned, I tried compiling the example with the release flag, just to check if timing would play a role here.
I didn't find anything, but this took me 14m 19s to compile it. Maybe you should narrow which modules you require from the windows crate.

Apart from that, have you tried running the NVDA COM fixing tool? I don't see how it could solve this issue, but trying it costs nothing.

[mwcampbell]

I can still reproduce a variation on this with the current version of the hello_world example if I comment out the COM initialization code. Specifically, with NVDA 2021.2 on Windows 10 build 19043.1288, if NVDA is already running when the program starts, the button is announced, but sometimes the window title is not announced, and if you press NVDA+F1, you can see that the button is identified as an MSAA object, not native UIA. Moving the focus away from the window and then back fixes it. I don't know how I'm going to deal with this now that I'm trying to integrate AccessKit into druid-shell. Having druid-shell initialize COM with the MTA might be too opinionated, as it might interfere with integrating third-party components that require an STA. But having all applications have this platform-specific workaround is probably not acceptable either.

[mwcampbell]

@DataTriny I have a guess about why you can't reproduce this issue. Is the tabtip.exe process running on your PC? In the Windows 10 task manager, I believe it's called something like "Windows Touch and Handwriting Panel". As the name implies, it implements the touch keyboard on touch-enabled PCs. I have a touchscreen laptop, though I never use the touchscreen. If I kill that process, then the problem goes away. If I restart that process, then the problem comes back. I don't know what happens if you manually start that process on a non-touch-enabled PC. If you want to try it, it's in "C:\Program Files\Common Files\Microsoft Shared\Ink" (on English Windows).

Anyway, I think I'm going to work around this by having accesskit_windows::Manager check whether COM is initialized on the thread that owns the window, and if not, initialize COM using the MTA (multi-threaded apartment).

[DataTriny]

@mwcampbell How on earth were you able to find out?

If I manually start this service, I get the same kind of behavior you described here (no window title and IAccessible objects in the NVDA console). Looks like something odd is happening on Microsoft's side.

BTW, if I comment out lines 231-232 of hello_world.rs then the program always exits with this error:

error: process didn't exit successfully: `D:\repositories\accesskit\target\debug\examples\hello_world.exe` (exit code: 0xc000041d)

The error code is apparently defined like so in ntstatus.h:

// MessageId: STATUS_FATAL_USER_CALLBACK_EXCEPTION
//
// MessageText:
//
// An unhandled exception was encountered during a user callback.
//
#define STATUS_FATAL_USER_CALLBACK_EXCEPTION ((NTSTATUS)0xC000041DL)

[mwcampbell]

@DataTriny To answer your question about how I found out, I was exposed to tabtip.exe while I was on the Windows accessibility team at Microsoft. I never studied that particular program's code, but I had general awareness about what it does, and that it makes some use of accessibility APIs. Then yesterday, I noticed that the example program was getting WM_GETOBJECT messages even when I wasn't running a screen reader (I have enough sight to read the screen up close), and, remembering that my laptop has a touchscreen, I suspected tabtip.exe.