[SDL] SDL_EVDEV_do_text_input() may be too eager to find errors

[SirCodesAlot]

Hello,

It took me awhile to figure out how to post here. This is a duplicate of a bug report I filed last night; https://bugzilla.libsdl.org/show_bug.cgi?id=3545. I'm posting it here in the hopes more eyes have a chance to look at it, and provide feedback/help. Thanks!

I've ran into an issue where I successfully receive SDL_KEY[UP,DOWN] events but not SDL_TEXTINPUT or SDL_TEXTEDITING. In my case the code in SDL_EVDEV_do_text_input() is returning early (on error) prior to calling SDL_SendKeyboardText(). I'm running on the RaspberryPi 3, without X11.

In SDL_EVDEV_do_text_input() there is a condition to check keysyms with a type value below 0xf0, then subtract 0xf0 from type. Without understanding the purpose of this code, I disabled it, recompiled, and I'm getting correct SDL_TEXTINPUT events. I'm going to guess that my hack/fix is going to be problematic in some other environment, but after some initial testing it looks like everything is running fine in my setup.

My setup is a little different than what is recommended in docs/README-raspberrypi.md, so its quite possible this is the source of my pain. I'm hoping someone more familiar with udev may be able to jump in and help identify if this is the case.

My setup:

* SDL2-2.0.5 on the RaspberryPi 3, running the 4.4.34-v7+ kernel, and udev 215.
* Cross-compiled using gcc 4.8.3.
* I'm building SDL with support for udev, and dbus. IME, ibus, and fcitx are disabled.
* I've disabled X11 support when building SDL. I'm using the GLES2 and VideoCore libraries.
* In docs/README-raspberrypi.md it recommends a way to modify a fully built out raspbian image. My approach is different. I'm building up my own rootfs. It's also based on raspbian packages, but only those that I explicitly want.
* I'm not installing any X11 packages (like those suggested in docs/README-raspberrypi.md).


I've attached a tar.gz to the bug report that contains a number of files that may be interesting to review. Direct link to the file: https://bugzilla.libsdl.org/attachment.cgi?id=2658

* hack.patch - My SDL_evdev.c modifications. Seriously, this is a hack for debug/test.
* keyboardhell.cpp - Test code to loop on keyboard events from SDL_PollEvent().
* capture.log - keyboardhell execution on RPi after applying hack.patch. Note the following keyboard input sequence: 'l','s',<ENTER>,<ESC>. Also of interest SDL_EVDEV_mute_keyboard() does not appear to mute tty.
* sdl_configure.sh - Script used to ./configure SDL build.
* sdl.log - SDL build log.
* OTHER - all other logs were captured on the RPi.

-Rob

[SirCodesAlot]

After reducing my hacky patch to the minimum number of changes needed to to fix my setup, I uncovered another bug in SDL_EVDEV_is_console() that I accidentally fixed. https://bugzilla.libsdl.org/show_bug.cgi?id=3546


When calling ioctl(fd, KDGKBTYPE, &type) in SDL_EVDEV_is_console(), we declare type as an 'int'. This should be a 'char'. The subsequent syscall, and kernel code, only writes the lower byte of the word.

See: http://lxr.free-electrons.com/source/drivers/tty/vt/vt_ioctl.c?v=4.4#L399

ucval = KB_101;
ret = put_user(ucval, (char __user *)arg);

I've observed intermittent behavior related to this, and I can force an error condition by using an int initialized to 0xFFFFFFFF. The resulting ioctl will set type to 0XFFFFFF02, and the conditional return in SDL_EVDEV_is_console() will fail.

Recommend changing to char, or masking off unused bits.

See attached patch for suggested fix.

-Rob