Microphone disabled when accessing local installation remotely

[CodeShadower]

Context: I run llama.cpp on a desktop and the other servers on my laptop, then I access Amica through my smartphone.

Laptop

• Amica server 127.0.0.1:3000
• alltalk_tts 0.0.0.0:7851
• whisper.cpp 0.0.0.0:8080

Desktop

• llama.cpp 0.0.0.0:5000

Smartphone (client)
From my smartphone on the same LAN, I can access the Amica server, I can write to Amica and Amica is talking back (Yeah!).
However, the microphone icon next to the text input is greyed out on the Amica UI.

In another browser tab, on the same smartphone, I can connect to Amica online and there, the microphone icon is active.

What do I have to do to be able to use Amica's microphone button?

[slowsynapse]

It worked last time I checked, maybe you can activate development on mobile browser and see what errors come up? We'll check our side too.

[CodeShadower]

It worked last time I checked, maybe you can activate development on mobile browser and see what errors come up? We'll check our side too.

Thank you, I'll have to activate developer tools remotely as I cannot activate the developer mode on my smartphone's browser directly.

I'll also try to get a second laptop to check from there whether the mic is active or not to narrow it down. I'll post back the results.

[slowsynapse]

Thanks @CodeShadower

[CodeShadower]

So, I finally managed to test it with a second laptop having developer tools available, here's the result:>

And in the same Firefox browser, when I open another tab and navigate to amica.arbius.ai I immediately get this pop-up:

And this is the developer tools output for the amica.arbius.ai website:

[slowsynapse]

Thanks! This should be very helpful, I noticed it is giving a VAD error, did you try Chrome?

[slowsynapse]

From Dev:

For this issue, we have faced the same error before (vad error : getUserMedia isn't implemented in this browser) and the problem come from i have disable the microphone access for that specific site, I have fixed it by manually enable the microphone for that site. From the Issue Owner Context, I think you might have to allow microphone access for "predator.local:3000/" in firefox : https://support.psychologytoday.com/allowing-access-to-your-camera-and-microphone-in-firefox-mobile/tablet

[slowsynapse]

Let me know if this helps @CodeShadower

[CodeShadower]

Thank you ver much @slowsynapse for pursuing the investigation on your end!

Here are my observations:

Enabling the microphone was one of my first reflexes, but it is simply not possible as this option and more are disabled in the settings, as well in Firefox as in Chromium. However now, after lokking closer, this leads me to suspect that the fact that I have an unsecured site (http://) instead of secured one (https://) as you publish online with https://amica.arbius.ai, even if its local, the browsers seem to block certain settings because of that. I tested by entering the IP address and also by entering the local host name, same result:

Based on that, I searched for a way to bypass the security blocking for the local address. For Chrome there is this flag which allows exactly that: chrome://flags/#unsafely-treat-insecure-origin-as-secure

One has to define the exact URL, in my case: http://192.168.1.10:3000 or http://predator.local:3000

This finally had the effect to allow your application scripts to run properly and allow me to allow the microphone usage within the browser! 😄

Now, I needed to find the option for Firefox Android and this is trickyer. Firefox Stable doesn't allow about:config, so I downloaded Focus and the nightly version of it. I made sure also that Android allowed the microphone on OS level, but somehow this doesn't seem to work.

So I switched to Brave using the same settings as for Chromium, but note that flags on the android version must be called with "chrome:flags" instead of "chrome://flags" as for the PC version. 😉

This finally worked!!! So, I leave Firefox for that particular use case, Brave is fine. Case closed for me!
(and now Amica won't stop talking!!! 🤣 🤣 🤣)

Next! I'll come to you with a multimode issue. But that'll be during the holidays! 😉

Have great and relaxing holidays! 🎄 ⛄ 🍾

[mikewalshchicago]

Rather than hack the security of your remote browser, I think it is a better idea to set up an nginx reverse proxy with a ssl certificate. getting to amica should be as simple as the following in your /etc/nginx/sites-available/default:

server {
listen 443 ssl;
ssl_certificate /etc/nginx/tls/localhost.crt;
ssl_certificate_key /etc/nginx/tls/localhost.key;
location / {
proxy_pass http://127.0.0.1:3000/;
}
}

This works for me, however amica is having trouble reaching the LLM (llama.cpp) this way. I should think that the amica application itself it interfacing with the llama-server, so the settings in the webui should remain http://127.0.0.1:8080. This is not the case though. The remote UI reports TypeError: Failed to fetch. Thinking that maybe the LLM is engaged directly from the browser, I tried to change the url to the IP of the server, but that didn't work either. I even set up a server in nginx listening on 8888 and forwarding to http://localhost:8080, but that didn't work either.

I feel that amica and the LLM should be able to communicate with each other independent of the nginx proxy, so I still think this the way to go despite my failure up to this point. Perhaps someone with more insight can explain how the amica application queries the local LLM and that would help me understand why the chatbot doesnt work through a browser running on a remote computer

[mikewalshchicago]

FWIW: I managed to get the mic working and probably the camera (jury's still out on that, because she says she can see me but she can't count my fingers) on my smartphone using nginx as a ssl proxy for both amica and the llama.cpp server. What I have below will work, but the the webpage resets every minute or so, which makes any conversation frustrating. I'm guessing the connection is being reset because the backend is sending replies in http. Even though that's expected and my understanding is that nginx will take care of this, i've only done this on static websites before so there is probably something lacking in my config

server {
listen 3003 ssl;
listen [::]:3003 ssl;
ssl_certificate /etc/nginx/tls/localhost.crt;
ssl_certificate_key /etc/nginx/tls/localhost.key;
server_name (IP of the server};

   location / {
       proxy_pass http://localhost:3000/;
       proxy_ssl_verify off;
       proxy_buffering off;
       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;

   }

}

server {
listen 8088 ssl;
listen [::]:8088 ssl;
ssl_certificate /etc/nginx/tls/localhost.crt;
ssl_certificate_key /etc/nginx/tls/localhost.key;
server_name {IP of the server};

   location / {
       proxy_pass http://localhost:8080/;
       proxy_ssl_verify off;
       proxy_buffering off;
       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;

   }

}