Skip to content

Problem with accessibility in R 4.2.0 and 4.2.1.

5 messages · Andrew Hart, Jonathan Godfrey, Tomas Kalibera

#
Dear Andrew, Jonathan,

I had a closer look and tried to improve accessibility in Rgui, please 
see below. I would be grateful for feedback.

Rgui supports three cursor types, which can be selected via Edit/GUI 
preferences/Cursor blink.

The default is "partial", but for screen readers, please use "full". You 
can change the selection in the menu and then "Save..." to save it into 
your Rconsole file. If you already have the file, the corresponding 
selection is "cursor_blink = Full".

The "full" cursor is implemented as the standard Windows "caret" and 
this is what screen readers can see. Once you set this cursor as the 
default in your Rconsole file and re-start Rgui, but before you start 
using the console, please switch focus out and back in (e.g. press 
Alt+TAB twice). This helps NVDA detect the characters under the cursor 
in already released versions of R. Please start the screen reader before 
starting Rgui.

I found that the "full" cursor implementation has a number of problems: 
in some situation it disappears when it shouldn't, in some the other way 
around. I thought this was confusing the screen reader, so I fixed most 
of these cases.

However, the true cause was that Rgui didn't create the cursor right 
when it got focus the first time. Luckily NVDA is open-source, so one 
can read and modify the source code to find out. I've added a 
work-around to Rgui, which is used only with the "full" cursor, because 
the "partial" cursor confuses the screen reader too much to be usable, 
anyway. So, after this fix, one doesn't have to do that focus out+in trick.

The improvements are in R-devel (revision 83482 or newer). I would be 
grateful if you could test it, so that it could be improved further (or 
reverted if it actually turned out worse). Particularly if you find a 
problem reproducible with NVDA, that should be something I could 
diagnose and improve, as I have the sources.

I've been testing with NVDA and I'd be curious about the impact on JAWS.
Rgui doesn't work with Narrator.

Thanks,
Tomas
On 9/22/22 23:15, Andrew Hart via R-devel wrote:
#
HI Tomas,

Thanks a lot for not letting this go. It is truly appreciated. I had 
been using Rterm directly as Jonathan had suggested since we discussed 
this a number of months ago on the R-devel list. However, about a week 
and a half ago I accidentally launched Rgui for R 4.2.2 (which I 
installed around the end of October) and was surprised when I could 
actually  use it like I could use the pre-4.2 versions of R! I have been 
using it for a little more than a week now and was intending to write to 
you, but you beat me with this message.
The accessibility of Rgui 4.2.2 seems very similar to R 4.1.2 (which I 
still have on my system). In contrast, Rgui 4.2.1 is more or less 
unusable. I was wanting to ask you if perhaps something got changed in R 
4.2.2? Nothing jumped out at me in the release notes, but I could easily 
have overlooked something.

After reading this message, I went and checked the cursor blink setting 
in my 4.2.2 installation and it is indeed set to partial. You're right 
in that occasionally JAWS loses the cursor and the ability to read the R 
window. However, simply pressing the <enter> key while Rgui has the 
focus fixes this. It seems that drawing a new command prompt on a new 
line sets JAWS straight and I am able to keep working. I haven't used 
4.1.2 for a while, but I think it had the same issue. I assumed that 
this was caused by moving from Windows 7 to Windows 10, since I never 
encountered this kind of issue in windows 7. I'll try out full cursor 
and see if that makes a difference in 4.2.2.

Also, I'll download the development snapshot and try it out. I'll let 
you know how I get on. Please excuse me if I don't get to it 
immediately; things are a bit crazy at work at the moment and it is that 
time of the year too!

Cheers,
Andrew.
On 20/12/2022 19:33, Tomas Kalibera wrote:
#
Hi Andrew,

thanks a lot for your testing and I am looking forward to what you find 
out about the fixes in R-devel.

You asked about changes between 4.2.1 and 4.2.2 relevant to this. I 
don't think you overlooked anything, I looked now again at the source 
code diff and I didn't find anything relevant. The Rgui console was 
fixed to work with Alt+sequences (tilde on Italian keyboard). GraphApp 
was fixed to use the correct font charset for UTF-8 in dialog boxes. 
Search and replace was fixed in the script editor. Invalid write was 
fixed with printing very long lines.

None of those changes should impact the behavior of the caret in the 
console window. I would not be surprised if the behavior with the screen 
reader was unpredictable/random, certainly inconsequential to whether we 
had R 4.2.1 or 4.2.2. Hopefully the changes now in R-devel would make it 
more reliable. If not, we can try to improve it further.

Thanks for testing with JAWS, I am impressed it can get anything out 
from Rgui with the partial cursor. NVDA cannot, it allows navigating 
using left arrow/right arrow over the line (after focus out+in in R 
4.2.2), but it thinks that the characters on the line are all spaces.

In either case, I think using the partial cursor with screen readers 
doesn't make sense, it could only confuse the screen reader application. 
If some people preferred a wider cursor than "full", we could add a 
"wide full" cursor, that would be quite easy. If some people preferred 
less blinking or no blinking at all, they can already set that up 
system-wide in Windows for the "full" (or potential "wide full") cursor.

Cheers
Tomas
On 12/21/22 17:12, Andrew Hart wrote:
#
Hello both,

I so seldom use the RGUI that I hand't noticed it was behaving nicely in 4.2.2 which has been running on my laptop for a long time!

I confirm the JAWS experience is back to what I expected prior to 4.2.0 and assure you that I do nothing to alter the default settings. I do think maintaining the usefulness of the GUI has merit because that is what a new R user will end up trying once they discover RStudio's inaccessibility.

I really do not understand how/why switching a cosmetic element of a cursor should lead to changes in the screen reader performance. I'm sure the size, shape, and colour of a mouse pointer  doesn't affect it's performance although I would expect it to have an impact on a user's performance.

Rather, I assumed that the change in the cosmetics of the cursor was the symptom of an underlying change that had multiple impacts, one of which was the disconnect with the screen reader. 

Did the underlying development toolbox undergo any version changes from ( <4.2.0 ) to (4.2.0...4.2.1) to (4.2.2) ??

Great that normal transmission has resumed.

Jonathan


-----Original Message-----
From: Tomas Kalibera <tomas.kalibera at gmail.com> 
Sent: Thursday, 22 December 2022 6:53 am
To: Andrew Hart <ahart at dim.uchile.cl>; Jonathan Godfrey <A.J.Godfrey at massey.ac.nz>
Cc: R-devel at r-project.org
Subject: Re: [Rd] Problem with accessibility in R 4.2.0 and 4.2.1.

Hi Andrew,

thanks a lot for your testing and I am looking forward to what you find 
out about the fixes in R-devel.

You asked about changes between 4.2.1 and 4.2.2 relevant to this. I 
don't think you overlooked anything, I looked now again at the source 
code diff and I didn't find anything relevant. The Rgui console was 
fixed to work with Alt+sequences (tilde on Italian keyboard). GraphApp 
was fixed to use the correct font charset for UTF-8 in dialog boxes. 
Search and replace was fixed in the script editor. Invalid write was 
fixed with printing very long lines.

None of those changes should impact the behavior of the caret in the 
console window. I would not be surprised if the behavior with the screen 
reader was unpredictable/random, certainly inconsequential to whether we 
had R 4.2.1 or 4.2.2. Hopefully the changes now in R-devel would make it 
more reliable. If not, we can try to improve it further.

Thanks for testing with JAWS, I am impressed it can get anything out 
from Rgui with the partial cursor. NVDA cannot, it allows navigating 
using left arrow/right arrow over the line (after focus out+in in R 
4.2.2), but it thinks that the characters on the line are all spaces.

In either case, I think using the partial cursor with screen readers 
doesn't make sense, it could only confuse the screen reader application. 
If some people preferred a wider cursor than "full", we could add a 
"wide full" cursor, that would be quite easy. If some people preferred 
less blinking or no blinking at all, they can already set that up 
system-wide in Windows for the "full" (or potential "wide full") cursor.

Cheers
Tomas
On 12/21/22 17:12, Andrew Hart wrote:
#
Hello Jonathan,

Rgui uses GraphApp as toolbox, but that is part of R source code and 
I've included into my summary of changes between 4.2.1 and 4.2.2, and I 
couldn't find anything that I think could impact the screen reader on 
the Rgui console. The changes in GraphApp are included also in NEWS, 
many of the fixes in Rgui are in fact in GraphApp. GraphApp then 
directly uses the Windows Graphics Device Interface (GDI). I am not 
aware of any such changes even between 4.2.0 and 4.2.1, either. So I 
have no clue why differences in screen reader performance were observed.

I've now tried with JAWS as well, the default installation, 40 minute 
trial. I did this experiment:

1. start Rgui
2. type "hello"
3. navigate back using left arrow

On R 4.0.5, 4.1.3, 4.2.1, 4.2.2, R-devel with the defaults (so 
"partial") cursor, JAWS worked fine and the same in all versions. It 
could read the letters back when I was navigating over them. It worked 
also with the "full" cursor. I didn't find any difference. I noticed 
that JAWS increased the cursor blink rate, which sighted people could 
find annoying.

Trying the same with NVDA.? "partial" cursor didn't work with any R 
version, the letters could be read when typed, but not when navigated 
back. The "full" cursor worked fine in R-devel (so with my recent 
fixes). In the released R versions I tested, I could make the "full" 
cursor work by switching focus (before step 3) by going to the "Edit" 
menu and back, e.g. "Alt+E followed by ESC followed by ESC".

So it looks like my fix wasn't needed for JAWS, but only for NVDA.

Of course my experiment is rather limited, if you or anyone else could 
test more scenarios and report any problems with the current R-devel 
Rgui (and the "full" cursor), it would be a great help and we can try to 
fix those.

Regarding changes that did happen to the R sources in the past, which 
clearly could impact the screen reader performance, I see R 2.12.0 has 
in NEWS: "The Rgui console now has an optional blinking caret (cursor) 
to assist screen readers for the visually impaired." Before this change, 
Rgui only used its custom drawn cursor, about which it didn't tell 
Windows, and hence Windows couldn't tell the screen readers. After this 
change, one could choose also "partial" (default) and "full" cursor. 
"partial" included the custom drawn cursor together with the standard 
Windows cursor ("caret") Windows knows about. "Full" cursor included 
only the standard Windows cursor. This was an addition by Duncan 
Murdoch, which allowed the navigation to work. "None" (none blinking) 
cursor in R 4.2.2 is like the original custom drawn cursor and neither 
NVDA nor JAWS with that can read the characters back when they are 
navigated over: they cannot detect the cursor.

I don't see any changes in the implementation of the cursor since the 
original Duncan's code for 2.12.0. I fixed some name clashes in 76745 in 
June 2019 (so R 4.0.0), but that should be completely benign.

However, what I did see was non-standard behavior of the standard 
Windows cursor in Rgui when switching windows, going to menu, starting 
Rgui, etc. I've fixed most but not (yet) all of those. It may be that 
users have ran into intermittent problems with the screen readers as a 
consequence of such actions (sometimes they lead not only to visually 
loosing the Windows cursor, but also to the cursor really disappearing 
so that Windows didn't know about it anymore). It is well possible that 
this also affected JAWS. I know for sure it did affect NVDA, based on 
which I did the current fixes.

Best
Tomas
On 12/21/22 20:33, Jonathan Godfrey wrote: