MCU/FreeRTOS support

[kdschlosser]

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
I have been working with micro controllers that have attached touch screen displays and writing a binding for a GUI framework to MicroPython. One of the things I am striving to provide to the users is fast prototyping. Currently a user is able to write Python code and it will run on a micro controller for creating a GUI to be shown on an attached display. The binding is also able to be compiled to run on macOS and also Linux and this allows a user to be able to develop without needing to use an MCU with an attached display. While the ability to test a GUI on a PC does make things easier it has a downside, no hardware related bits are available. So NO GPIO access and things like that. This would make sense seeing as how the binding is running on a PC and not on the actual MCU.

What I am wanting to do is allow the user to be able to develop without the need to have a display attached to the MCU. Using the RFB protocol would be the best solution because just about every OS available has some form of VNC client software available. Ideally I would like to make thing easier by using an already existing library. The only hangup I am having is all the libraries I have come across handle to actual receiving and transmitting of the data. This is something that I would not use because the MCU I am using doesn't have the same API for networking as Linux et al. has. The library would be used to handle the reading of the incoming data and formatting the outgoing data but the actual sending and receiving would be done using code that I would write. This could be done easily if there are functions that would handle the reading of the data and creating the packets needed for sending.

The MCU I am working with uses an Xtensa 32-bit LX7 processor with 2 cores @ 240mhz. Depending on how it's configured it can have 2mb-8mb of RAM and 4mb-32mb of flash based program storage.

The first thing to hurdle is getting the library to compile using a gcc like compiler that is made for these specific MCU's. I don't know enough about CMAKE to know if adding it to my projects CMAKE build will carry over using a specific compiler and also how to set option flags that would normally be done by passing them in the build command.

The processor that I am using is dual core and I have the ability to set specific code to run on specific cores. I see that this library supports pthread but unfortunately this is not available. FreeRTOS is what is used to handle "threads". while the functionality of how the threading works is pretty close to pthread the API is completely different. I did come across a library that is a wrapper around FreeRTOS to provide a pthread API. I have not tested this to see if it will work. I am pretty sure there is going to be things that I will need to modify to get it to run on the MCU I am using. If there were macros that could be changed by the user to point to a different function for thing like creating a mutex and creating a thread this would make you library very flexible.

Describe alternatives you've considered
I have looked at quite a few different VNC server libraries as a possible solution. Your library is the most complete in terms of much of the RFB protocol API is supported and the way it is written also would make it easier to make alterations that would allow it to be run on a micro controller.

Additional context
Now that I am thinking about it providing macros to handle using encryption libraries other than the ones directly supported by the library would be of added benefit as a lot of MCUs have hardware accelerators for doing the encrypting and decrypting.

[bk138]

You can build without threading support, the rest is probably reseaching cmake and FreeRTOS. If your efforts are successful, let me know; would be happy to merge in some documentation w.r.t. embedded systems!

[kdschlosser]

That would really make the library truly cross platform.
What I am wanting to do is bare metal compiling so no OS. Not really an embedded system.

question tho. are there functions that are public that can encode and decode the VNC packets without having to use any of the built in connection code. The network related API for the ESP32 doesn't follow POSIX standards, not 100% anyway. so I would only use the library to encode and decode the packets. the library would handle reading the data that I would feed into it via a function call and it would work like it would normally would when doing that. for sending messages I would only need the library to build the packets when transmitting.

[bk138]

I guess your best bet is to write/get-from-a-third-party shims for BSD socket functions; there might be other issues as well. But at least documenting those here in public would be nice.

[kdschlosser]

I am still crossing the bridge to get it to compile. It appears the only way to set the config options is I have to add the project as a sub directory. That is the only way the environment will carry over to it.

There is no way to just use the packet encoding/decoding portions of the library? I could write wrapper code for the network functions to make it work.

[kdschlosser]

I am actually pretty damned close to getting it to compile. There are a few issues however.

You have some macro definitions that are redefinition's. you don't have any guards in place to see if they are already defined. I am thinking you may want to do this...

you have INADDR_NONE being defined in both the server and the client. Not sure why but they are bumping heads. They also bump heads with it being defined in lwip (light weight IP stack) which is what is used by the esp32. Nice thing about lwip is it has posix compat headers YAY!! and between that and also the posix compat header that can be gotten for FreeRTOS that is what is allowing me to compile your library on the MCU. I did have to write a new cmake script to handle ti but that wasn't too big a deal to do.

I am going to manually add the guard for that macro and see if there is anything else that causes problems. at the moment that is all there appears to be.

[bk138]

If you got a simple PR with additional header guards, I'll happily merge this.

[kdschlosser]

I got sidetracked on something else and I have not put a guard in for the one error I am getting. Once I have it compiling I will submit a PR for any changes to the code I have made,

[kdschlosser]

OK so I am making headway. so far a simple include solves the issue. In include/rfb/rfbproto.h there is this code...

#ifndef INADDR_NONE
#define                INADDR_NONE     ((in_addr_t) 0xffffffff)
#endif

and it's awesome that there is a guard in place to check to see if it is already defined. Issue is this macro gets set before sys/socket.h is included anywhere in libvncserver. sys/socket.h gets included after the macro gets set and the issue is there is no guard in inet.h which is where the macro gets set a second time.

There are a couple of ways to handle this.

1. include sys/socket.h in include/rfb/rfbproto.h prior to the macro getting set.
2. tell the user that they must include sys/socket.h in their project prior to including any part of libvncserver

Personally I believe that option 1 is the best way to go about it because it is pretty much a gurantee never have anyone open an issue should they not read the docs or they missed the line where it informs them of option 2 above. It's not going to cause any kind of an issue to include sys/socket.h before that macro gets set as it is something that ends up being used in libvncserver anyhow.

I changed the lines seen below which are located in include/rfb/rfbproto.h

#if defined(WIN32)
typedef int8_t rfbBool;
#include <sys/timeb.h>
#include <winsock2.h>
#endif

to read

#if defined(WIN32)
typedef int8_t rfbBool;
#include <sys/timeb.h>
#include <winsock2.h>
#else
#include <sys/socket.h>
#endif

and that solved that issue.

Next issue is a bit more complex and it is one that is going to take me a bit to figure out how to solve. Perhaps you might be able to help me out with the best solution.

This is the issue.

In a multitude of places errno.h is included in the esp32 build. The esp32 SDK (ESP-IDF) has created replacement headers for some things so it aligns with the posix API. Not all things have been created which is why there is a need to use the compat headers from lwip and also from another library that is a wrapper around FreeRTOS to provide a posix API for creating threads. Now here is the issue.

Includes are handled so when an include is wrapped in double quotes the way the compiler does the search is local path from where the include is taking place and then it searches the paths that are provided using the -I compiler argument. When an include is wrapped in gt and le the compiler searches the paths provided by the -I compiler argument first and then it looks for includes at the system level.

Se here is the issue.... lwip provides an errno.h as does the FreeRTOS wrapper and the ESP32 SDK provides one as well. since the file exists in 3 locations the first one that the compiler comes across when searching the paths is going to be the one that it is going to use. The issue is not all of the needed definitions exist in a single one of those files. The only way I can think to handle this is to create my own errno.h file and include each of the other 3 files using the abolsute path to each file and wrapping each one in double quotes. Now I am able to collect the paths programmatically using cmake, no issue there. what I do not know how to do is to create the file using cmake. This is something that you might be able to assist me in doing. In order to do this I would be using part of the ESP-IDF build system to collect the paths to the files. I think it would be better if code was written that wouldn't be tied specifically to the ESP-IDF build system. If there is a way to search the include paths supplied to cmake to locate header files that are the same name and create a new header file if multiples are found and include the multiples in the new header file and set the path to the generated file so it will be what is used when included.. I think that is the proper way to go about it.

It would take me days to figure out how to do that with cmake so it would do a proper recursive search so the paths and filename match properly and create the same path and filename.

[bk138]

My 2 cents here: For the time being, don't bother to automate this with cmake, let humans do it. I think a section in the README describing the very special steps for the errno.h situation on esp32 would be enough.

[kdschlosser]

OK so I am getting down to the wire with this...

couple of errors I am not sure how to fix.,...

/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/sockets.c: In function 'rfbProcessNewConnection':
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/sockets.c:480:19: error: storage size of 'rlim' isn't known
480 |     struct rlimit rlim;
|                   ^~~~
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/sockets.c:509:8: error: implicit declaration of function 'getrlimit' [-Werror=implicit-function-declaration]
509 |     if(getrlimit(RLIMIT_NOFILE, &rlim) < 0)
|        ^~~~~~~~~
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/sockets.c:509:18: error: 'RLIMIT_NOFILE' undeclared (first use in this function)
509 |     if(getrlimit(RLIMIT_NOFILE, &rlim) < 0)
|                  ^~~~~~~~~~~~~
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/sockets.c:509:18: note: each undeclared identifier is reported only once for each function it appears in
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/sockets.c:480:19: warning: unused variable 'rlim' [-Wunused-variable]
480 |     struct rlimit rlim;
|                   ^~~~

/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncclient/listen.c: In function 'listenForIncomingConnections':
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncclient/listen.c:91:18: error: implicit declaration of function 'wait4'; did you mean 'wait'? [-Werror=implicit-function-declaration]
91 |     while ((pid= wait4(-1, &status, WNOHANG, (struct rusage *)0))>0);
|                  ^~~~~
|                  wait

/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/stats.c: In function 'messageNameClient2Server':
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/stats.c:83:42: error: format '%X' expects argument of type 'unsigned int', but argument 4 has type 'uint32_t' {aka 'long unsigned int'} [-Werror=format=]
83 |         snprintf(buf, len, "cli2svr-0x%08X", type);
|                                       ~~~^   ~~~~
|                                          |   |
|                                          |   uint32_t {aka long unsigned int}
|                                          unsigned int
|                                       %08lX
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/stats.c: In function 'encodingName':
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/stats.c:157:38: error: format '%X' expects argument of type 'unsigned int', but argument 4 has type 'uint32_t' {aka 'long unsigned int'} [-Werror=format=]
157 |         snprintf(buf, len, "Enc(0x%08X)", type);
|                                   ~~~^    ~~~~
|                                      |    |
|                                      |    uint32_t {aka long unsigned int}
|                                      unsigned int
|                                   %08lX

/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/httpd.c: In function 'validateString':
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncserver/httpd.c:661:22: error: array subscript has type 'char' [-Werror=char-subscripts]
661 |         if (!isalnum(*ptr) && *ptr != '_' && *ptr != '.'
|                      ^~~~

/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncclient/vncviewer.c: In function 'rfbGetClient':
/vnc_compile_test/lvgl_micropython/lib/libvncserver/src/libvncclient/vncviewer.c:343:36: warning: cast between incompatible function types from 'rfbBool (*)(rfbClient *, int,  int)' {aka 'signed char (*)(struct _rfbClient *, int,  int)'} to 'void (*)(struct _rfbClient *, int,  int)' [-Wcast-function-type]
343 |   client->HandleKeyboardLedState = (HandleKeyboardLedStateProc)DummyPoint;
|                                    ^

[bk138]

This is probably all stuff not in FreeRTOS. I recommend commenting out vigorously and getting stuff to build, take notes, then clean up later.

PS: Also, you probably don't need libvncclient, so errors there can be ignored.