Syndria98/quibot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Compare commits

base: Syndria98:main
Branches Tags
Syndria98:main Syndria98:master
Syndria98:latest
..
compare: Syndria98:5b9216e764699d5e616dc95f347c8c3d567d6294
Branches Tags
Syndria98:main Syndria98:master
Syndria98:latest

1 Commits
main ... 5b9216e764

Author SHA1 Message Date
BinarySandia04
5b9216e764 Apk compilation 2026-06-18 14:09:54 +02:00
89 changed files with 8500 additions and 5101 deletions
Expand all files Collapse all files

4
.gitea/workflows/build-apk.yml
View File

@@ -2,7 +2,7 @@ name: Build APK
on: on:
push: push:
branches: [ main ] branches: [ master ]
jobs: jobs:
build: build:
@@ -66,6 +66,6 @@ jobs:
name: Latest Build name: Latest Build
overwrite_files: true overwrite_files: true
files: | files: |
dist/app-release.apk dist/app-release-apk.zip
env: env:
GITEA_TOKEN: ${{ secrets.GITEA }} GITEA_TOKEN: ${{ secrets.GITEA }}

4
.gitea/workflows/build.yml
View File

@@ -3,7 +3,7 @@ name: Build
on: on:
push: push:
branches: branches:
- main - master
jobs: jobs:
build-web: build-web:
@@ -73,8 +73,6 @@ jobs:
with: with:
name: backend name: backend
path: backend/backend.zip path: backend/backend.zip
release: release:
runs-on: docker runs-on: docker

128
AGENTS.md
View File

@@ -2,17 +2,16 @@
## Overview ## Overview
QuiBot is an educational robotics platform (UPC/UNE collaboration) consisting of a programmable robot with color-block recognition, gesture control, stepper motors, RGB LED eyes, and multiple input methods (web dashboard, Android voice app). The codebase comprises two independent application layers communicating via HTTP JSON APIs. QuiBot is an educational robotics platform (UPC/UNE collaboration) consisting of a programmable robot with color-block recognition, gesture control, stepper motors, RGB LED eyes, and multiple input methods (web dashboard, Android voice app). The codebase comprises four independent application layers communicating via HTTP JSON APIs.
``` ```
[quibot-web Nuxt SPA] ──HTTP──> [backend Express] [quibot-web Nuxt SPA] ──HTTP──> [backend Express] ──HTTP──> [raspi FastAPI (Pi)]
│ │ │ │
▼ ├──▶ Raspberry Pi (port 8000) — motor/audio endpoints ▼ ▼
[apk Expo RN app] ──HTTP──> ├──▶ LLM (llamacpp) [apk Expo RN app] ──HTTP──> (same backend) [Python hardware drivers]
└──▶ TTS (Piper)
``` ```
**Tech stack**: TypeScript/Express | Nuxt 4/Vue 3 | Expo/React Native **Tech stack**: Python (pigpio, FastAPI) | TypeScript/Express | Nuxt 4/Vue 3 | Expo/React Native
**No database**: All state is in-memory, file-based (`/tmp/quibot-audio/`), or localStorage. **No database**: All state is in-memory, file-based (`/tmp/quibot-audio/`), or localStorage.
--- ---
@@ -21,22 +20,30 @@ QuiBot is an educational robotics platform (UPC/UNE collaboration) consisting of
``` ```
quibot/ quibot/
├── backend/ # Express server (port 5000), proxy to Pi + LLM/TTS ├── raspi/ # Raspberry Pi brain — Python, controls hardware
│ ├── main.py # FastAPI server (port 8000) on the Pi
│ ├── quibot.py # Main program: block/gesture threads (like Arduino QuiBot.ino)
│ ├── motion.py # Stepper class, homing, line-following, high-level tasks
│ ├── gesture.py # PAJ7620U2 gesture sensor (I2C, polled at 50ms)
│ ├── blocks.py # TCS34725 color sensor + servo block ejection
│ ├── eyes.py # WS2811 LED matrix (128 LEDs, pigpio waveforms, breathing animation)
│ ├── pins.py # BCM GPIO pin map for all hardware
│ └── tests/ # Manual diagnostic scripts (not automated)
├── backend/ # Local Express server (port 3000), proxy to Pi
│ ├── src/ │ ├── src/
│ │ ├── index.ts # Express entry: CORS, JSON parser, /health, routes │ │ ├── index.ts # Express entry: CORS, JSON parser, /health
│ │ ├── config.ts # Env config: raspi, PIPER_URL, LLAMA_CPP_URL, PORT │ │ ├── config.ts # Env config: RASPBERRY_PI_HOST, PORT, QUIBOT_TOKEN
│ │ ├── routes/router.ts # Mounts all controllers │ │ ├── routes/router.ts # Mounts all controllers
│ │ ├── services/raspi.service.ts # Axios proxy layer to Pi endpoints │ │ ├── services/raspi.service.ts # Axios proxy layer to Pi FastAPI
│ │ └── controllers/ │ │ └── controllers/
│ │ ├── motor.controller.ts # Motor step/stop/upload │ │ ├── motor.controller.ts # Motor step/stop/upload
│ │ ├── audio.controller.ts # Audio file lifecycle (incoming/locked/processed) │ │ ├── audio.controller.ts # Audio file lifecycle (incoming/locked/processed)
│ │ ├── command.controller.ts # POST /commands proxy to raspi /run │ │ ├── command.controller.ts # POST /commands proxy to raspi /run
│ │ ├── settings.controller.ts # GET/PUT /settings runtime config │ │ └── settings.controller.ts # GET/PUT /settings runtime config
│ │ └── tts.controller.ts # TTS synthesis via Piper
│ └── dist/ # Compiled output (generated) │ └── dist/ # Compiled output (generated)
├── quibot-web/ # Nuxt 4 dashboard SPA ├── quibot-web/ # Nuxt 4 dashboard SPA
│ ├── app/app.vue # Single-page control panel: block queue, D-pad, eye controls, gesture log │ ├── app/app.vue # Single-page control panel: block queue, D-pad, eye controls, gesture log
│ ├── server/api/ # Nitro server routes proxying to backend Express │ ├── server/api/ # Nitro server routes proxying to raspi
│ │ ├── motor/step/[direction].post.ts │ │ ├── motor/step/[direction].post.ts
│ │ └── motor/stop.post.ts │ │ └── motor/stop.post.ts
│ ├── nuxt.config.ts # Runtime config: QUIBOT_BASE_URL, QUIBOT_TOKEN │ ├── nuxt.config.ts # Runtime config: QUIBOT_BASE_URL, QUIBOT_TOKEN
@@ -54,9 +61,7 @@ quibot/
--- ---
## Raspberry Pi Layer (remote, port 8000) ## Raspberry Pi Layer (`raspi/`)
The Raspberry Pi runs a lightweight HTTP server exposing hardware control endpoints. The `raspi/` source directory is no longer part of this repository — it lives on the Pi itself.
**Hardware target**: Raspberry Pi Zero 2W controlling a robot with: **Hardware target**: Raspberry Pi Zero 2W controlling a robot with:
- 5 NEMA-style stepper motors (wheels x2, arms x2, syringe) via A4988/TB600 drivers in STEP/DIR mode - 5 NEMA-style stepper motors (wheels x2, arms x2, syringe) via A4988/TB600 drivers in STEP/DIR mode
@@ -69,7 +74,7 @@ The Raspberry Pi runs a lightweight HTTP server exposing hardware control endpoi
- Hall-effect endstops on GPIOs 12, 16, 17 - Hall-effect endstops on GPIOs 12, 16, 17
- Optional: I2S audio amp (MAX98357A) + mic (SPH0645) - Optional: I2S audio amp (MAX98357A) + mic (SPH0645)
### Hardware source files (on Pi) ### Key files
- **`pins.py`** — BCM GPIO pin numbering for every component (STEP, DIR, EN pins, I2C lines, endstops, LED_DATA on GPIO26) - **`pins.py`** — BCM GPIO pin numbering for every component (STEP, DIR, EN pins, I2C lines, endstops, LED_DATA on GPIO26)
- **`motion.py`** — `Stepper` class with AccelStepper-style acceleration profiling via `pigpio.gpio_trigger()`. 5 motor instances (`wheel_R`, `wheel_L`, `arm_R`, `arm_L`, `syringe`). Continuous stepper daemon thread (`_stepper_loop`) at ~100Hz. Homing routines read Hall-effect endstops. Line-following with proportional correction on TCRT5000 values via ADS1115. - **`motion.py`** — `Stepper` class with AccelStepper-style acceleration profiling via `pigpio.gpio_trigger()`. 5 motor instances (`wheel_R`, `wheel_L`, `arm_R`, `arm_L`, `syringe`). Continuous stepper daemon thread (`_stepper_loop`) at ~100Hz. Homing routines read Hall-effect endstops. Line-following with proportional correction on TCRT5000 values via ADS1115.
@@ -99,7 +104,7 @@ The Raspberry Pi runs a lightweight HTTP server exposing hardware control endpoi
## Backend Layer (`backend/`) ## Backend Layer (`backend/`)
**Role**: Express.js HTTP server providing frontend/mobile API, proxying hardware commands to the Raspberry Pi, and managing TTS/LLM integration. **Role**: Express.js HTTP proxy sitting between frontend/mobile and the Raspberry Pi's FastAPI server. Token passthrough, no business logic.
### Configuration (`.env`, loaded by `config.ts`) ### Configuration (`.env`, loaded by `config.ts`)
@@ -108,19 +113,15 @@ The Raspberry Pi runs a lightweight HTTP server exposing hardware control endpoi
| `RASPBERRY_PI_HOST` | `http://raspberrypi.local` | Pi API URL | | `RASPBERRY_PI_HOST` | `http://raspberrypi.local` | Pi API URL |
| `RASPBERRY_PI_PORT` | `8000` | Pi API port | | `RASPBERRY_PI_PORT` | `8000` | Pi API port |
| `QUIBOT_TOKEN` | `MY_SECRET_TOKEN` | Auth token for all Pi endpoints | | `QUIBOT_TOKEN` | `MY_SECRET_TOKEN` | Auth token for all Pi endpoints |
| `PORT` | `5000` | Backend listen port | | `PORT` | `3000` | Backend listen port |
| `PIPER_URL` | `''` | Piper TTS service URL |
| `LLAMA_CPP_URL` | `''` | LLM inference service URL |
| `LLAMA_API_KEY` | `''` | LLM API key |
| `LLAMA_PREAMBLE` | `''` | Path or content for LLM preamble |
### Architecture ### Architecture
``` ```
index.ts → Express app, CORS, JSON parser, /health endpoint index.ts → Express app, CORS, JSON parser, /health endpoint
routes/router.ts → Mounts all controllers under /motor, /audio, /commands, /settings, /tts routes/router.ts → Mounts all controllers under /motor, /audio, /commands, /settings
config.ts → Mutable getter/setter env vars (runtime update via PUT /settings) config.ts → Mutable getter/setter env vars (runtime update via PUT /settings)
raspi.service.ts → Axios proxy methods for Pi endpoints + multipart file upload handling raspi.service.ts → Axios proxy methods for every Pi endpoint + multipart file upload handling
``` ```
### Controllers ### Controllers
@@ -129,7 +130,6 @@ raspi.service.ts → Axios proxy methods for Pi endpoints + multipart file uplo
- **`audio.controller.ts`** — `GET /audio/incoming`, `POST /audio/lock/:filename`, `/unlock/:filename`, `/cancel/:filename`, `/process/:filename`. All proxy to raspi audio file lifecycle endpoints. - **`audio.controller.ts`** — `GET /audio/incoming`, `POST /audio/lock/:filename`, `/unlock/:filename`, `/cancel/:filename`, `/process/:filename`. All proxy to raspi audio file lifecycle endpoints.
- **`command.controller.ts`** — `POST /commands { task }` → proxied to raspi `/run?task=...&token=...` - **`command.controller.ts`** — `POST /commands { task }` → proxied to raspi `/run?task=...&token=...`
- **`settings.controller.ts`** — `GET /settings` returns config; `PUT /settings` updates `raspberryPi.host`, `raspberryPi.port`, `token` at runtime. - **`settings.controller.ts`** — `GET /settings` returns config; `PUT /settings` updates `raspberryPi.host`, `raspberryPi.port`, `token` at runtime.
- **`tts.controller.ts`** — `POST /tts { text, lang }` → Synthesizes audio via Piper TTS service. Saves WAV files to `/tmp/quibot-audio/tts/`.
### Build/Run ### Build/Run
@@ -150,7 +150,7 @@ node dist/index.js # Or use tsx/nodemon for dev
| Key | Default | Purpose | | Key | Default | Purpose |
|-----|---------|---------| |-----|---------|---------|
| `QUIBOT_BASE_URL` | `http://quibot:8000` | Base URL for backend Express (or Pi) | | `QUIBOT_BASE_URL` | `http://quibot:8000` | Base URL for raspi FastAPI |
| `QUIBOT_TOKEN` | `MY_SECRET_TOKEN` | Auth token | | `QUIBOT_TOKEN` | `MY_SECRET_TOKEN` | Auth token |
### UI Panels (`app/app.vue` — single-file SPA, 1369 lines) ### UI Panels (`app/app.vue` — single-file SPA, 1369 lines)
@@ -170,8 +170,8 @@ node dist/index.js # Or use tsx/nodemon for dev
| Method | Path | Description | | Method | Path | Description |
|--------|------|-------------| |--------|------|-------------|
| POST | `/api/motor/stop` | Proxies to backend Express `/motor/stop` | | POST | `/api/motor/stop` | Proxies to raspi `/motor/stop` |
| POST | `/api/motor/step/:direction` | Proxies to backend Express `/motor/step/forward\|backwards` | | POST | `/api/motor/step/:direction` | Proxies to raspi `/motor/step/forward\|backwards` |
**Note**: The frontend also calls `POST /api/eye/shape`, `/api/eye/color`, `/api/eye/on`, `/api/eye/off`, `/api/gesture/on`, `/api/gesture/off` — server routes for these may need to be created (frontend references them but they don't have explicit server handlers yet). **Note**: The frontend also calls `POST /api/eye/shape`, `/api/eye/color`, `/api/eye/on`, `/api/eye/off`, `/api/gesture/on`, `/api/gesture/off` — server routes for these may need to be created (frontend references them but they don't have explicit server handlers yet).
@@ -221,26 +221,7 @@ CI: `build-apk.yml` runs expo prebuild, decodes keystore from secrets, builds si
## Complete API Reference ## Complete API Reference
### Backend Express (`backend/`) — port 5000 ### Raspberry Pi FastAPI (`raspi/main.py`) — port 8000
| Method | Path | Body | Description |
|--------|------|------|-------------|
| GET | `/health` | — | Returns settings object |
| POST | `/commands` | `{ task }` | Proxy to raspi `/run` |
| POST | `/motor/step/forward` | — | Motor forward proxy to Pi |
| POST | `/motor/step/backward` | — | Motor backward proxy to Pi (maps to `/backwards`) |
| POST | `/motor/stop` | — | Motor stop proxy to Pi |
| POST | `/motor/upload` | multipart file | Audio upload via multer → proxied to Pi |
| GET | `/audio/incoming` | — | List incoming audio files from Pi |
| POST | `/audio/lock/:filename` | — | Lock audio file on Pi |
| POST | `/audio/unlock/:filename` | — | Unlock audio file on Pi |
| POST | `/audio/cancel/:filename` | — | Cancel locked audio on Pi |
| POST | `/audio/process/:filename` | — | Mark processed on Pi |
| GET | `/settings` | — | Returns config |
| PUT | `/settings` | `{ raspberryPi: { host, port }, token }` | Update runtime config |
| POST | `/tts` | query: `text`, `lang` | Synthesize speech via Piper TTS |
### Raspberry Pi HTTP Server (remote, port 8000)
| Method | Path | Params/Body | Description | | Method | Path | Params/Body | Description |
|--------|------|-------------|-------------| |--------|------|-------------|-------------|
@@ -257,6 +238,24 @@ CI: `build-apk.yml` runs expo prebuild, decodes keystore from secrets, builds si
**Auth**: Query parameter `token` matching `QUIBOT_TOKEN` env var (default: `MY_SECRET_TOKEN`). **Auth**: Query parameter `token` matching `QUIBOT_TOKEN` env var (default: `MY_SECRET_TOKEN`).
### Backend Express (`backend/`) — port 3000
| Method | Path | Body | Description |
|--------|------|------|-------------|
| GET | `/health` | — | Returns settings object |
| POST | `/commands` | `{ task }` | Proxy to raspi `/run` |
| POST | `/motor/step/forward` | — | Motor forward proxy |
| POST | `/motor/step/backward` | — | Motor backward proxy (maps to raspi `/backwards`) |
| POST | `/motor/stop` | — | Motor stop proxy |
| POST | `/motor/upload` | multipart file | Audio upload via multer in-memory buffer |
| GET | `/audio/incoming` | — | List incoming audio files |
| POST | `/audio/lock/:filename` | — | Lock audio file |
| POST | `/audio/unlock/:filename` | — | Unlock audio file |
| POST | `/audio/cancel/:filename` | — | Cancel locked audio |
| POST | `/audio/process/:filename` | — | Mark processed |
| GET | `/settings` | — | Returns config |
| PUT | `/settings` | `{ raspberryPi: { host, port }, token }` | Update runtime config |
--- ---
## Command Flow Examples ## Command Flow Examples
@@ -267,11 +266,9 @@ User clicks "Forward" in D-pad
→ $fetch('/api/motor/step/forward', { method: 'POST' }) → $fetch('/api/motor/step/forward', { method: 'POST' })
→ Nuxt Nitro route: server/api/motor/step/[direction].post.ts → Nuxt Nitro route: server/api/motor/step/[direction].post.ts
→ $fetch(config.quibotBaseUrl + '/motor/step/forward', { query: { token } }) → $fetch(config.quibotBaseUrl + '/motor/step/forward', { query: { token } })
→ Backend Express /motor/step/forward → raspi FastAPI /motor/step/forward
→ raspi.service.motorStepForward() → motor_step("forward") in daemon thread
→ Pi FastAPI /motor/step/forward?token=... → step_motor(200, DIR, 1ms pulses)
→ motor_step("forward") in daemon thread on Pi
→ step_motor(200, DIR, 1ms pulses)
``` ```
### Block Processing (internal to Pi) ### Block Processing (internal to Pi)
@@ -281,18 +278,8 @@ Child inserts colored block → quibot.py task_read_blocks() polls distance sens
→ Manhattan distance classification against color lookup table → Manhattan distance classification against color lookup table
→ RED: eyes_turn_on(EYES_FW, DARK_RED, 2) → RED: eyes_turn_on(EYES_FW, DARK_RED, 2)
_execute_action(task_move_to, CROSSING) _execute_action(task_move_to, CROSSING)
→ enable_wheels(ON) → follow_line_loop(speed) (proportional on TCRT5000) → enable_wheels(ON) → follow_line_loop(speed) (proportional on TCRT5000)
→ After action: servo_move_to(EJECT_POSITION) → After action: servo_move_to(EJECT_POSITION)
```
### TTS Synthesis
```
User triggers speech in web UI
→ POST /tts?text=hello&lang=ca&token=...
→ Backend Express tts.controller.ts
→ piperService.synthesize() → Piper TTS service
→ WAV file saved to /tmp/quibot-audio/tts/{uuid}.wav
→ Returns audioUrl + filename
``` ```
### APK → Audio Upload ### APK → Audio Upload
@@ -330,9 +317,8 @@ User toggles mode in web UI
## Testing ## Testing
Tests reside on the Pi alongside the hardware source code, not in this repository. All tests in `raspi/tests/` are **manual diagnostic scripts** (not automated frameworks). Each test is run independently by uncommenting the desired function call at the bottom of the file. Requirements:
Requirements:
- `sudo pigpiod -s 1` daemon running - `sudo pigpiod -s 1` daemon running
- Python venv activated with hardware dependencies installed - Python venv activated with hardware dependencies installed
- Pi connected to robot hardware - Pi connected to robot hardware
@@ -351,8 +337,8 @@ Requirements:
1. **Token auth** is always a query parameter matching `QUIBOT_TOKEN` (default: `MY_SECRET_TOKEN`) 1. **Token auth** is always a query parameter matching `QUIBOT_TOKEN` (default: `MY_SECRET_TOKEN`)
2. **No database** — use filesystem for persistence, localStorage for web client state 2. **No database** — use filesystem for persistence, localStorage for web client state
3. **Backend proxies Pi endpoints** — motor/audio commands forwarded via raspi.service.ts 3. **Backend is a dumb proxy** — no business logic, just forwards HTTP requests with token passthrough
4. **Motor commands are fire-and-forget** — motor runs in daemon thread on Pi until `/motor/stop` 4. **Motor commands are fire-and-forget** — motor runs in daemon thread until `/motor/stop`
5. **Audio lifecycle**: incoming → locked (claim) → processed OR unlocked (release) / cancelled 5. **Audio lifecycle**: incoming → locked (claim) → processed OR unlocked (release) / cancelled
6. **Eyes breathing** runs continuously at MIN_BR(80)-MAX_BR(170) brightness in background 6. **Eyes breathing** runs continuously at MIN_BR(80)-MAX_BR(170) brightness in background
7. **`quibot.py` owns block/gesture autonomy** — blocks are processed internally on the Pi without backend/web involvement 7. **`quibot.py` owns block/gesture autonomy** — blocks are processed internally on the Pi without backend/web involvement

9
apk/app.json
View File

@@ -32,6 +32,7 @@
}, },
"plugins": [ "plugins": [
"expo-router", "expo-router",
"@react-native-voice/voice",
[ [
"expo-av", "expo-av",
{ {
@@ -49,14 +50,6 @@
"backgroundColor": "#000000" "backgroundColor": "#000000"
} }
} }
],
[
"expo-build-properties",
{
"android": {
"usesCleartextTraffic": true
}
}
] ]
], ],
"experiments": { "experiments": {

666
apk/app/index.tsx
View File

@@ -1,4 +1,4 @@
import { Audio, InterruptionModeAndroid, InterruptionModeIOS } from "expo-av"; import Voice from "@react-native-voice/voice";
import { router, useFocusEffect } from "expo-router"; import { router, useFocusEffect } from "expo-router";
import { useCallback, useEffect, useRef, useState } from "react"; import { useCallback, useEffect, useRef, useState } from "react";
import Svg, { Path } from "react-native-svg"; import Svg, { Path } from "react-native-svg";
@@ -6,6 +6,7 @@ import {
ActivityIndicator, ActivityIndicator,
Alert, Alert,
KeyboardAvoidingView, KeyboardAvoidingView,
NativeModules,
Platform, Platform,
Pressable, Pressable,
ScrollView, ScrollView,
@@ -26,45 +27,22 @@ function formatDuration(durationMs: number) {
.padStart(2, "0")}`; .padStart(2, "0")}`;
} }
function buildMimeType(uri: string) {
const extension = uri.split(".").pop()?.split("?")[0]?.toLowerCase();
switch (extension) {
case "wav":
return "audio/wav";
case "caf":
return "audio/x-caf";
case "webm":
return "audio/webm";
case "mp3":
return "audio/mpeg";
default:
return "audio/m4a";
}
}
function buildFileExtension(uri: string) {
return uri.split(".").pop()?.split("?")[0]?.toLowerCase() || "m4a";
}
export default function RecorderScreen() { export default function RecorderScreen() {
const [backendUrl, setBackendUrl] = useState(""); const [backendUrl, setBackendUrl] = useState("");
const [authToken, setAuthToken] = useState(""); const [authToken, setAuthToken] = useState("");
const [fieldName, setFieldName] = useState("file");
const [locale, setLocale] = useState<Locale>("ca"); const [locale, setLocale] = useState<Locale>("ca");
const [strings, setStrings] = useState(() => getStrings("ca")); const [strings, setStrings] = useState(() => getStrings("ca"));
const [recording, setRecording] = useState<Audio.Recording | null>(null);
const [recordingUri, setRecordingUri] = useState<string | null>(null); const [transcript, setTranscript] = useState("");
const [recordingMs, setRecordingMs] = useState(0); const [interimTranscript, setInterimTranscript] = useState("");
const [statusMessage, setStatusMessage] = useState(""); const [statusMessage, setStatusMessage] = useState("");
const [responsePreview, setResponsePreview] = useState(""); const [responsePreview, setResponsePreview] = useState("");
const [llmResponseText, setLlmResponseText] = useState(""); const [isSending, setIsSending] = useState(false);
const [transcriptionText, setTranscriptionText] = useState("");
const [isUploading, setIsUploading] = useState(false);
const [isHolding, setIsHolding] = useState(false); const [isHolding, setIsHolding] = useState(false);
const [isPlaying, setIsPlaying] = useState(false); const [listeningMs, setListeningMs] = useState(0);
const recordingRef = useRef<Audio.Recording | null>(null); const startRef = useRef<number>(0);
const soundRef = useRef<Audio.Sound | null>(null); const timerRef = useRef<ReturnType<typeof setInterval> | null>(null);
const listeningActiveRef = useRef(false);
const refreshSettings = useCallback(() => { const refreshSettings = useCallback(() => {
let isMounted = true; let isMounted = true;
@@ -79,7 +57,6 @@ export default function RecorderScreen() {
setBackendUrl(settings.backendUrl); setBackendUrl(settings.backendUrl);
setAuthToken(settings.authToken); setAuthToken(settings.authToken);
setFieldName(settings.fieldName);
setLocale(settings.language); setLocale(settings.language);
setStrings(getStrings(settings.language)); setStrings(getStrings(settings.language));
} catch { } catch {
@@ -99,415 +76,187 @@ export default function RecorderScreen() {
useFocusEffect(refreshSettings); useFocusEffect(refreshSettings);
useEffect(() => { useEffect(() => {
if (!recording) { if (Platform.OS === "web" || !NativeModules.Voice) {
return; return;
} }
const interval = setInterval(() => { Voice.onSpeechStart = (e) => {
void recording.getStatusAsync().then((status) => { console.log("Voice.onSpeechStart", e);
if (typeof status.durationMillis === "number") { setTranscript("");
setRecordingMs(status.durationMillis ?? 0); setInterimTranscript("");
} setStatusMessage(strings.recording);
}); startRef.current = Date.now();
}, 250); timerRef.current = setInterval(() => {
const elapsed = Date.now() - startRef.current;
return () => { setListeningMs(elapsed);
clearInterval(interval); }, 250);
}; };
}, [recording]);
useEffect(() => { Voice.onSpeechEnd = (e) => {
return () => { console.log("Voice.onSpeechEnd", e);
if (recording) { listeningActiveRef.current = false;
void recording.stopAndUnloadAsync().catch(() => undefined); if (timerRef.current) {
clearInterval(timerRef.current);
timerRef.current = null;
} }
}; };
}, [recording]);
useEffect(() => { Voice.onSpeechResults = (e) => {
console.log("Voice.onSpeechResults", e);
const values = (e as unknown as { value: string[] }).value;
if (values && values.length > 0) {
const last = values[values.length - 1];
setTranscript(last);
}
};
Voice.onSpeechPartialResults = (e) => {
console.log("Voice.onSpeechPartialResults", e);
const values = (e as unknown as { value: string[] }).value;
if (values && values.length > 0) {
const last = values[values.length - 1];
setInterimTranscript(last || "");
}
};
Voice.onSpeechError = (e) => {
console.log("Voice.onSpeechError", e);
const message = (e as unknown as { error?: { message: string } }).error?.message || "Speech recognition error";
setStatusMessage(message);
if (timerRef.current) {
clearInterval(timerRef.current);
timerRef.current = null;
}
setListeningMs(0);
listeningActiveRef.current = false;
Alert.alert(strings.recordingFailedTitle, message);
};
return () => { return () => {
void unloadSound(); Voice.removeAllListeners();
if (timerRef.current) {
clearInterval(timerRef.current);
timerRef.current = null;
}
}; };
}, []); }, []);
async function unloadSound() { useEffect(() => {
if (soundRef.current) { return () => {
try { if (listeningActiveRef.current && Platform.OS !== "web" && NativeModules.Voice) {
await soundRef.current.stopAsync(); Voice.stop().catch(() => undefined);
await soundRef.current.unloadAsync(); listeningActiveRef.current = false;
} catch (err) {
console.log("[TTS] Error unloading sound:", err);
} }
soundRef.current = null; };
} }, []);
setIsPlaying(false);
}
async function speakWithAudio(audioUrl: string, backendBase: string) {
if (!audioUrl) return false;
await unloadSound();
async function startListening() {
try { try {
await Audio.setAudioModeAsync({
allowsRecordingIOS: false,
playsInSilentModeIOS: true,
interruptionModeAndroid: InterruptionModeAndroid.DoNotMix,
interruptionModeIOS: InterruptionModeIOS.DoNotMix,
shouldDuckAndroid: true,
staysActiveInBackground: false,
});
} catch (err) {
console.log("[TTS] Audio mode error:", err);
}
try {
const fullUrl = audioUrl.startsWith("http")
? audioUrl
: `${backendBase.replace(/\/+$/, "")}/${audioUrl.replace(/^\/+/, "")}`;
console.log("[TTS] Loading audio from:", fullUrl);
setIsPlaying(true);
setStatusMessage(strings.playing);
const { sound } = await Audio.Sound.createAsync(
{ uri: fullUrl },
{ shouldPlay: true, volume: 1.0 },
(status) => {
if (status.isLoaded && status.didJustFinish) {
console.log("[TTS] Audio playback finished");
void unloadSound();
}
},
);
soundRef.current = sound;
const status = await sound.getStatusAsync();
const durationMs = status.isLoaded ? (status.durationMillis ?? 0) : 0;
console.log("[TTS] Playing audio, duration:", durationMs, "ms");
return true;
} catch (err) {
console.log("[TTS] Audio playback error:", err);
setIsPlaying(false);
return false;
}
}
async function speakSequentially(texts: string[]) {
if (texts.length === 0) return;
const trimmedUrl = backendUrl.trim().replace(/\/+$/, "");
for (let i = 0; i < texts.length; i++) {
const text = texts[i];
if (!text || !text.trim()) continue;
try {
setStatusMessage(strings.playing);
console.log("[TTS] Generating TTS audio for text:", text.substring(0, 50));
const localeLang = locale === "ca" ? "ca" : "en";
const ttsParams = new URLSearchParams({
text: text.trim(),
language: localeLang,
});
if (authToken.trim()) {
ttsParams.append("token", authToken.trim());
}
const ttsUrl = `${trimmedUrl}/tts?${ttsParams.toString()}`;
const ttsResponse = await fetch(ttsUrl, { method: "POST" });
if (!ttsResponse.ok) {
const errText = await ttsResponse.text();
console.log("[TTS] TTS endpoint error:", ttsResponse.status, errText);
continue;
}
const ttsData = await ttsResponse.json();
if (!ttsData.audioUrl) {
console.log("[TTS] No audioUrl in response:", ttsData);
continue;
}
const played = await speakWithAudio(ttsData.audioUrl, trimmedUrl);
if (!played) {
setStatusMessage(strings.uploadFailed);
}
if (i < texts.length - 1) {
await new Promise((r) => setTimeout(r, 800));
}
} catch (err) {
console.log("[TTS] speakSequentially error:", err);
}
}
}
async function speak(text: string) {
const texts = [text].filter(Boolean);
await speakSequentially(texts);
}
async function startRecording() {
try {
await unloadSound();
setTranscriptionText("");
setResponsePreview(""); setResponsePreview("");
setLlmResponseText(""); setTranscript("");
setRecordingUri(null); setInterimTranscript("");
const localeCode =
const permission = await Audio.requestPermissionsAsync(); locale.includes("ca")
? "ca-ES"
if (!permission.granted) { : locale.includes("es")
setStatusMessage(strings.micPermissionDenied); ? "es-ES"
Alert.alert( : "en-US";
strings.micAccessRequiredTitle, setIsHolding(true);
strings.micAccessRequiredMsg, if (Platform.OS === "web") {
); console.log("Voice not available on web");
Alert.alert("Not supported", "Speech recognition is only available on mobile devices. Open the app on Android or iOS.");
setIsHolding(false);
return; return;
} }
if (!NativeModules.Voice) {
await Audio.setAudioModeAsync({ Alert.alert("Not supported", "Speech recognition module not found. Make sure the app is built with native modules.");
allowsRecordingIOS: true, setIsHolding(false);
interruptionModeAndroid: InterruptionModeAndroid.DoNotMix, return;
interruptionModeIOS: InterruptionModeIOS.DoNotMix, }
playsInSilentModeIOS: true, await Voice.start(localeCode);
shouldDuckAndroid: true, listeningActiveRef.current = true;
staysActiveInBackground: false,
});
const result = await Audio.Recording.createAsync(
Audio.RecordingOptionsPresets.HIGH_QUALITY,
);
recordingRef.current = result.recording;
setRecording(result.recording);
setRecordingMs(0);
setStatusMessage(strings.recording);
} catch (error) { } catch (error) {
setStatusMessage(strings.couldNotStartRecording); const msg = error instanceof Error ? error.message : "Failed to start speech recognition";
Alert.alert( console.error("Voice.start failed:", error);
strings.recordingFailedTitle, Alert.alert(strings.recordingFailedTitle, msg);
error instanceof Error ? error.message : "",
);
} }
} }
async function stopRecordingAndUpload() { async function stopListeningAndSend() {
if (!recordingRef.current) { setIsHolding(false);
const wasListening = listeningActiveRef.current;
if (!wasListening) {
setStatusMessage(transcript ? strings.voiceMessageSent : strings.readyToRecord);
return;
}
listeningActiveRef.current = false;
if (Platform.OS === "web") {
const finalText = (transcript + " " + interimTranscript).trim().replace(/\s+/g, " ");
if (finalText) {
await sendCommand(finalText);
} else {
setStatusMessage(strings.readyToRecord);
}
return; return;
} }
console.log("[APP] stopRecordingAndUpload called");
try { try {
const activeRecording = recordingRef.current; await Voice.stop();
const currentStatus = await activeRecording.getStatusAsync();
const durationMillis = currentStatus.durationMillis ?? 0;
await activeRecording.stopAndUnloadAsync(); const finalText = (transcript + " " + interimTranscript).trim().replace(/\s+/g, " ");
await Audio.setAudioModeAsync({
allowsRecordingIOS: false,
playsInSilentModeIOS: true,
});
const uri = activeRecording.getURI(); if (!finalText) {
recordingRef.current = null;
setRecording(null);
setRecordingMs(durationMillis);
if (!uri) {
setStatusMessage(strings.readyToRecord); setStatusMessage(strings.readyToRecord);
return; return;
} }
setRecordingUri(uri); await sendCommand(finalText);
setStatusMessage(strings.finishedUpload);
const trimmedUrl = backendUrl.trim().replace(/\/+$/, '');
const uploadUrl = trimmedUrl.endsWith('/audio/upload')
? trimmedUrl
: `${trimmedUrl}/audio/upload`;
if (uploadUrl) {
setIsUploading(true);
try {
const mimeType = buildMimeType(uri);
const extension = buildFileExtension(uri);
const formData = new FormData();
formData.append(fieldName.trim() || "file", {
name: `recording-${Date.now()}.${extension}`,
type: mimeType,
uri: uri,
} as never);
const headers: Record<string, string> = {};
if (authToken.trim()) {
headers.Authorization = `Bearer ${authToken.trim()}`;
}
const response = await fetch(uploadUrl, {
method: "POST",
headers,
body: formData,
});
const responseText = await response.text();
if (!response.ok) {
throw new Error(`${response.status}. ${responseText}`);
}
try {
const data = JSON.parse(responseText);
setResponsePreview(responseText.slice(0, 400));
const textsToSpeak: string[] = [];
if (data.transcription) {
setTranscriptionText(data.transcription);
}
if (data.llmResponse) {
setLlmResponseText(data.llmResponse);
textsToSpeak.push(data.llmResponse);
}
if (textsToSpeak.length > 0) {
setStatusMessage(strings.voiceMessageSent + ". " + strings.playing);
void speakSequentially(textsToSpeak);
} else {
setLlmResponseText("");
}
} catch (parseError) {
console.log("[APP] JSON parse failed:", parseError, "Response was:", responseText.substring(0, 200));
setResponsePreview(responseText.slice(0, 400));
setTranscriptionText("");
setLlmResponseText("");
}
setStatusMessage(strings.voiceMessageSent);
} catch (error) {
setStatusMessage(strings.uploadFailed);
Alert.alert(
strings.uploadFailed,
error instanceof Error ? error.message : "",
);
} finally {
setIsUploading(false);
}
} else {
setStatusMessage(strings.noBackendUrl);
setIsUploading(false);
}
} catch (error) { } catch (error) {
recordingRef.current = null; listeningActiveRef.current = false;
setRecording(null); const msg = error instanceof Error ? error.message : "Stop failed";
setStatusMessage(strings.stopFailedTitle); console.error("Voice.stop failed:", error);
Alert.alert( setStatusMessage(msg);
strings.stopFailedTitle,
error instanceof Error ? error.message : "",
);
} }
} }
async function handlePressIn() { async function sendCommand(text: string) {
if (isUploading) return; const trimmedUrl = backendUrl.trim().replace(/\/+$/, "");
setIsHolding(true); const commandUrl = trimmedUrl.endsWith("/commands")
await startRecording(); ? `${trimmedUrl}/text`
} : `${trimmedUrl}/commands/text`;
async function handlePressOut() { if (!commandUrl) {
if (!isHolding) return; setStatusMessage(strings.noBackendUrl);
setIsHolding(false);
await stopRecordingAndUpload();
}
async function uploadRecording(uriOverride?: string) {
const targetUri = uriOverride ?? recordingUri;
if (!targetUri) {
return;
}
const trimmedUrl = backendUrl.trim().replace(/\/+$/, '');
const uploadUrl = trimmedUrl.endsWith('/audio/upload')
? trimmedUrl
: `${trimmedUrl}/audio/upload`;
if (!uploadUrl) {
Alert.alert(strings.missingBackendUrlTitle, strings.missingBackendUrlMsg);
return; return;
} }
try { try {
setIsUploading(true); setIsSending(true);
setStatusMessage(strings.uploadingRecording); setStatusMessage(strings.uploadingRecording);
await unloadSound();
setTranscriptionText("");
setResponsePreview("");
setLlmResponseText("");
const mimeType = buildMimeType(targetUri); const headers: Record<string, string> = {
const extension = buildFileExtension(targetUri); "Content-Type": "application/json",
const formData = new FormData(); };
formData.append(fieldName.trim() || "file", {
name: `recording-${Date.now()}.${extension}`,
type: mimeType,
uri: targetUri,
} as never);
const headers: Record<string, string> = {};
if (authToken.trim()) { if (authToken.trim()) {
headers.Authorization = `Bearer ${authToken.trim()}`; headers.Authorization = `Bearer ${authToken.trim()}`;
} }
const response = await fetch(uploadUrl, { const response = await fetch(commandUrl, {
method: "POST", method: "POST",
headers, headers,
body: formData, body: JSON.stringify({ text }),
}); });
const responseText = await response.text(); const responseText = await response.text();
setResponsePreview(responseText.slice(0, 400));
setTranscript("");
setInterimTranscript("");
if (!response.ok) { if (!response.ok) {
throw new Error(`${response.status}. ${responseText}`); throw new Error(`${response.status}. ${responseText}`);
} }
try { setStatusMessage(strings.voiceMessageSent);
const data = JSON.parse(responseText);
setResponsePreview(responseText.slice(0, 400));
const textsToSpeak: string[] = [];
if (data.transcription) {
setTranscriptionText(data.transcription);
textsToSpeak.push(data.transcription);
}
if (data.llmResponse) {
setLlmResponseText(data.llmResponse);
textsToSpeak.push(data.llmResponse);
}
if (textsToSpeak.length > 0) {
setStatusMessage(strings.voiceMessageSent + ". " + strings.playing);
void speakSequentially(textsToSpeak);
} else {
setLlmResponseText("");
}
} catch {
setResponsePreview(responseText.slice(0, 400));
setTranscriptionText("");
setLlmResponseText("");
}
setStatusMessage(strings.uploadComplete);
} catch (error) { } catch (error) {
setStatusMessage(strings.uploadFailed); setStatusMessage(strings.uploadFailed);
Alert.alert( Alert.alert(
@@ -515,22 +264,28 @@ export default function RecorderScreen() {
error instanceof Error ? error.message : "", error instanceof Error ? error.message : "",
); );
} finally { } finally {
setIsUploading(false); setIsSending(false);
} }
} }
function handleSpeak() { async function handlePressIn() {
const texts = [transcriptionText, llmResponseText].filter(Boolean); if (isSending) return;
void speakSequentially(texts); await startListening();
}
async function handlePressOut() {
await stopListeningAndSend();
} }
const releaseLabel = t("releaseToStop", locale); const releaseLabel = t("releaseToStop", locale);
const holdLabel = t("holdToRecord", locale); const holdLabel = t("holdToRecord", locale);
const openSettingsLabel = t("openSettingsHint", locale); const openSettingsHint = t("openSettingsHint", locale);
const appTitleLabel = t("appTitle", locale); const appTitleLabel = t("appTitle", locale);
const recorderTitleLabel = t("recorderTitle", locale); const recorderTitleLabel = t("recorderTitle", locale);
const serverResponseLabel = t("serverResponse", locale); const serverResponseLabel = t("serverResponse", locale);
const displayText = interimTranscript || transcript;
return ( return (
<View style={styles.safeArea}> <View style={styles.safeArea}>
<KeyboardAvoidingView <KeyboardAvoidingView
@@ -569,20 +324,20 @@ export default function RecorderScreen() {
<View style={styles.panel}> <View style={styles.panel}>
<Text style={[styles.meterValueCentered, isHolding && { color: "#d04f2d" }]}> <Text style={[styles.meterValueCentered, isHolding && { color: "#d04f2d" }]}>
{formatDuration(recordingMs)} {formatDuration(listeningMs)}
</Text> </Text>
<Pressable <Pressable
disabled={isUploading} disabled={isSending}
onPressIn={handlePressIn} onPressIn={handlePressIn}
onPressOut={handlePressOut} onPressOut={handlePressOut}
style={[ style={[
styles.micButton, styles.micButton,
isHolding ? styles.holdingButton : styles.idleButton, isHolding ? styles.holdingButton : styles.idleButton,
isUploading && styles.buttonDisabled, isSending && styles.buttonDisabled,
]} ]}
> >
{isUploading ? ( {isSending ? (
<ActivityIndicator color="#fff6f3" size="large" /> <ActivityIndicator color="#fff6f3" size="large" />
) : ( ) : (
<Svg width="64" height="64" viewBox="0 0 24 24" fill="none"> <Svg width="64" height="64" viewBox="0 0 24 24" fill="none">
@@ -608,44 +363,23 @@ export default function RecorderScreen() {
)} )}
</Pressable> </Pressable>
<Text style={styles.statusText}>{statusMessage || strings.readyToRecord}</Text> <Text style={styles.statusText}>
{statusMessage || strings.readyToRecord}
</Text>
<Text style={styles.helperText}> <Text style={styles.helperText}>
{isHolding {isHolding
? releaseLabel ? releaseLabel
: backendUrl.trim() : backendUrl.trim()
? holdLabel ? holdLabel
: openSettingsLabel} : openSettingsHint}
</Text> </Text>
{transcriptionText ? ( {displayText && (
<View style={styles.transcriptionBox}> <View style={styles.transcriptBox}>
<Text style={styles.transcriptionLabel}>{strings.yourMessage}</Text> <Text style={styles.transcriptLabel}>{strings.serverResponse}</Text>
<Text style={styles.transcriptionText}>{transcriptionText}</Text> <Text style={styles.transcriptText}>{displayText}</Text>
</View> </View>
) : null} )}
{llmResponseText ? (
<View style={styles.llmResponseBox}>
<View style={styles.llmResponseHeader}>
<Text style={styles.llmResponseLabel}>{strings.aiReply}</Text>
<Pressable onPress={handleSpeak} style={styles.speakButton}>
<Svg width="20" height="20" viewBox="0 0 24 24" fill="none">
<Path d="M11 5L6 9H2v6h4l5 4V5z" fill="#13304a" />
<Path d="M15.5 8.5a5.5 5.5 0 0 1 0 7" stroke="#13304a" strokeWidth="2" strokeLinecap="round" />
<Path d="M18.5 5.5a9 9 0 0 1 0 13" stroke="#13304a" strokeWidth="2" strokeLinecap="round" />
</Svg>
</Pressable>
</View>
<Text style={styles.llmResponseText}>{llmResponseText}</Text>
</View>
) : null}
{responsePreview && !transcriptionText && !llmResponseText ? (
<View style={styles.responseBox}>
<Text style={styles.responseLabel}>{serverResponseLabel}</Text>
<Text style={styles.responseText}>{responsePreview}</Text>
</View>
) : null}
</View> </View>
</ScrollView> </ScrollView>
</KeyboardAvoidingView> </KeyboardAvoidingView>
@@ -736,14 +470,6 @@ const styles = StyleSheet.create({
backgroundColor: "#d04f2d", backgroundColor: "#d04f2d",
transform: [{ scale: 1.08 }], transform: [{ scale: 1.08 }],
}, },
micButtonText: {
color: "#fff6f3",
fontSize: 20,
fontWeight: "800",
},
recordingLabel: {
fontSize: 18,
},
buttonDisabled: { buttonDisabled: {
opacity: 0.45, opacity: 0.45,
}, },
@@ -759,6 +485,28 @@ const styles = StyleSheet.create({
lineHeight: 18, lineHeight: 18,
textAlign: "center", textAlign: "center",
}, },
transcriptBox: {
backgroundColor: "#f7f0e0",
borderRadius: 16,
gap: 6,
marginTop: 4,
padding: 14,
maxWidth: "100%",
},
transcriptLabel: {
color: "#13304a",
fontSize: 13,
fontWeight: "700",
textTransform: "uppercase",
textAlign: "center",
},
transcriptText: {
color: "#36475a",
fontSize: 16,
lineHeight: 22,
textAlign: "center",
fontWeight: "500",
},
responseBox: { responseBox: {
backgroundColor: "#f7f0e0", backgroundColor: "#f7f0e0",
borderRadius: 16, borderRadius: 16,
@@ -778,58 +526,4 @@ const styles = StyleSheet.create({
fontSize: 14, fontSize: 14,
lineHeight: 20, lineHeight: 20,
}, },
llmResponseBox: {
backgroundColor: "#e8f4e8",
borderRadius: 16,
gap: 6,
marginTop: 4,
padding: 14,
borderWidth: 1,
borderColor: "#b8d9b8",
},
llmResponseHeader: {
flexDirection: "row",
alignItems: "center",
justifyContent: "space-between",
},
llmResponseLabel: {
color: "#2a6a2a",
fontSize: 13,
fontWeight: "700",
textTransform: "uppercase",
textAlign: "center",
},
llmResponseText: {
color: "#2d4a2d",
fontSize: 16,
lineHeight: 24,
},
speakButton: {
backgroundColor: "#f7f0e0",
borderRadius: 20,
padding: 6,
borderWidth: 1,
borderColor: "#dccfb9",
},
transcriptionBox: {
backgroundColor: "#e8ecf4",
borderRadius: 16,
gap: 6,
marginTop: 4,
padding: 14,
borderWidth: 1,
borderColor: "#b8c9d9",
},
transcriptionLabel: {
color: "#1a4a6a",
fontSize: 13,
fontWeight: "700",
textTransform: "uppercase",
textAlign: "center",
},
transcriptionText: {
color: "#1f3a52",
fontSize: 16,
lineHeight: 24,
},
}); });

6
apk/lib/translations/index.ts
View File

@@ -38,9 +38,6 @@ export function en() {
releaseToStop: "Release your finger to stop recording and send the audio.", releaseToStop: "Release your finger to stop recording and send the audio.",
holdToRecord: "Hold the microphone button to record a voice message. Release to send it immediately.", holdToRecord: "Hold the microphone button to record a voice message. Release to send it immediately.",
openSettingsHint: "Open settings to add your backend URL before sending voice messages.", openSettingsHint: "Open settings to add your backend URL before sending voice messages.",
aiReply: "Quibot reply",
yourMessage: "Your message",
playing: "Playing audio...",
}; };
} }
@@ -82,9 +79,6 @@ export function ca() {
releaseToStop: "Allibera el dit per aturar l'enregistrament i enviar l'\u00e0udio.", releaseToStop: "Allibera el dit per aturar l'enregistrament i enviar l'\u00e0udio.",
holdToRecord: "Mant\u00e9s premut el micr\u00f2fon per enregistrar un missatge de veu. Allibera'l per enviar-lo immediatament.", holdToRecord: "Mant\u00e9s premut el micr\u00f2fon per enregistrar un missatge de veu. Allibera'l per enviar-lo immediatament.",
openSettingsHint: "Obre la configuraci\u00f3 per afegir l'URL del servidor abans d'enviar missatges de veu.", openSettingsHint: "Obre la configuraci\u00f3 per afegir l'URL del servidor abans d'enviar missatges de veu.",
aiReply: "Resposta del Quibot",
yourMessage: "El teu missatge",
playing: "Reproduint àudio...",
}; };
} }

3182
apk/package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

10
apk/package.json
View File

@@ -11,11 +11,11 @@
}, },
"dependencies": { "dependencies": {
"@react-native-async-storage/async-storage": "2.2.0", "@react-native-async-storage/async-storage": "2.2.0",
"@react-native-picker/picker": "2.11.1", "@react-native-picker/picker": "^2.11.4",
"expo": "~54.0.35", "@react-native-voice/voice": "^3.2.4",
"expo": "~54.0.33",
"expo-av": "~16.0.8", "expo-av": "~16.0.8",
"expo-build-properties": "~1.0.10", "expo-router": "~6.0.23",
"expo-router": "~6.0.24",
"expo-splash-screen": "~31.0.13", "expo-splash-screen": "~31.0.13",
"expo-status-bar": "~3.0.9", "expo-status-bar": "~3.0.9",
"react": "19.1.0", "react": "19.1.0",
@@ -23,7 +23,7 @@
"react-native": "0.81.5", "react-native": "0.81.5",
"react-native-safe-area-context": "~5.6.0", "react-native-safe-area-context": "~5.6.0",
"react-native-screens": "~4.16.0", "react-native-screens": "~4.16.0",
"react-native-svg": "15.12.1", "react-native-svg": "^15.15.5",
"react-native-web": "~0.21.0" "react-native-web": "~0.21.0"
}, },
"devDependencies": { "devDependencies": {

16
backend/.env.example
View File

@@ -6,18 +6,4 @@ RASPBERRY_PI_PORT=8000
QUIBOT_TOKEN=MY_SECRET_TOKEN QUIBOT_TOKEN=MY_SECRET_TOKEN
# Backend server config # Backend server config
PORT=5000 PORT=3000
# Piper TTS config (optional — local model)
PIPER_MODELS_DIR=./piper
PIPER_MODEL=./src/ca_ES-upc_ona-medium.onnx
# Remote Piper TTS service (alternative to local model)
PIPER_URL=
LLAMA_CPP_URL=https://ollama.epsem.aranroig.com/v1/chat/completitions
LLAMA_PREAMBLE=./prompts/preamble.md
LLAMA_API_KEY=your_api_key
# MCP server (Python FastMCP) — SSH-tunelled from remote machine
MCP_URL=http://localhost:5001

2
backend/.gitignore vendored
View File

@@ -2,5 +2,3 @@ node_modules/
dist/ dist/
.env .env
*.log *.log
quibot-audio-*.txt
**/quibot-audio-*.txt

687
backend/package-lock.json generated
View File

@@ -8,7 +8,6 @@
"name": "quibot-backend", "name": "quibot-backend",
"version": "1.0.0", "version": "1.0.0",
"dependencies": { "dependencies": {
"@modelcontextprotocol/sdk": "^1.29.0",
"axios": "^1.7.0", "axios": "^1.7.0",
"cors": "^2.8.5", "cors": "^2.8.5",
"dotenv": "^16.4.5", "dotenv": "^16.4.5",
@@ -467,388 +466,6 @@
"node": ">=18" "node": ">=18"
} }
}, },
"node_modules/@hono/node-server": {
"version": "1.19.14",
"resolved": "https://registry.npmjs.org/@hono/node-server/-/node-server-1.19.14.tgz",
"integrity": "sha512-GwtvgtXxnWsucXvbQXkRgqksiH2Qed37H9xHZocE5sA3N8O8O8/8FA3uclQXxXVzc9XBZuEOMK7+r02FmSpHtw==",
"license": "MIT",
"engines": {
"node": ">=18.14.1"
},
"peerDependencies": {
"hono": "^4"
}
},
"node_modules/@modelcontextprotocol/sdk": {
"version": "1.29.0",
"resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.29.0.tgz",
"integrity": "sha512-zo37mZA9hJWpULgkRpowewez1y6ML5GsXJPY8FI0tBBCd77HEvza4jDqRKOXgHNn867PVGCyTdzqpz0izu5ZjQ==",
"license": "MIT",
"dependencies": {
"@hono/node-server": "^1.19.9",
"ajv": "^8.17.1",
"ajv-formats": "^3.0.1",
"content-type": "^1.0.5",
"cors": "^2.8.5",
"cross-spawn": "^7.0.5",
"eventsource": "^3.0.2",
"eventsource-parser": "^3.0.0",
"express": "^5.2.1",
"express-rate-limit": "^8.2.1",
"hono": "^4.11.4",
"jose": "^6.1.3",
"json-schema-typed": "^8.0.2",
"pkce-challenge": "^5.0.0",
"raw-body": "^3.0.0",
"zod": "^3.25 || ^4.0",
"zod-to-json-schema": "^3.25.1"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"@cfworker/json-schema": "^4.1.1",
"zod": "^3.25 || ^4.0"
},
"peerDependenciesMeta": {
"@cfworker/json-schema": {
"optional": true
},
"zod": {
"optional": false
}
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/accepts": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-2.0.0.tgz",
"integrity": "sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==",
"license": "MIT",
"dependencies": {
"mime-types": "^3.0.0",
"negotiator": "^1.0.0"
},
"engines": {
"node": ">= 0.6"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/body-parser": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.3.0.tgz",
"integrity": "sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==",
"license": "MIT",
"dependencies": {
"bytes": "^3.1.2",
"content-type": "^2.0.0",
"debug": "^4.4.3",
"http-errors": "^2.0.1",
"iconv-lite": "^0.7.2",
"on-finished": "^2.4.1",
"qs": "^6.15.2",
"raw-body": "^3.0.2",
"type-is": "^2.1.0"
},
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/body-parser/node_modules/content-type": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
"integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/content-disposition": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-1.1.0.tgz",
"integrity": "sha512-5jRCH9Z/+DRP7rkvY83B+yGIGX96OYdJmzngqnw2SBSxqCFPd0w2km3s5iawpGX8krnwSGmF0FW5Nhr0Hfai3g==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/cookie-signature": {
"version": "1.2.2",
"resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.2.2.tgz",
"integrity": "sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg==",
"license": "MIT",
"engines": {
"node": ">=6.6.0"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/debug": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
"integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/express": {
"version": "5.2.1",
"resolved": "https://registry.npmjs.org/express/-/express-5.2.1.tgz",
"integrity": "sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw==",
"license": "MIT",
"dependencies": {
"accepts": "^2.0.0",
"body-parser": "^2.2.1",
"content-disposition": "^1.0.0",
"content-type": "^1.0.5",
"cookie": "^0.7.1",
"cookie-signature": "^1.2.1",
"debug": "^4.4.0",
"depd": "^2.0.0",
"encodeurl": "^2.0.0",
"escape-html": "^1.0.3",
"etag": "^1.8.1",
"finalhandler": "^2.1.0",
"fresh": "^2.0.0",
"http-errors": "^2.0.0",
"merge-descriptors": "^2.0.0",
"mime-types": "^3.0.0",
"on-finished": "^2.4.1",
"once": "^1.4.0",
"parseurl": "^1.3.3",
"proxy-addr": "^2.0.7",
"qs": "^6.14.0",
"range-parser": "^1.2.1",
"router": "^2.2.0",
"send": "^1.1.0",
"serve-static": "^2.2.0",
"statuses": "^2.0.1",
"type-is": "^2.0.1",
"vary": "^1.1.2"
},
"engines": {
"node": ">= 18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/finalhandler": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-2.1.1.tgz",
"integrity": "sha512-S8KoZgRZN+a5rNwqTxlZZePjT/4cnm0ROV70LedRHZ0p8u9fRID0hJUZQpkKLzro8LfmC8sx23bY6tVNxv8pQA==",
"license": "MIT",
"dependencies": {
"debug": "^4.4.0",
"encodeurl": "^2.0.0",
"escape-html": "^1.0.3",
"on-finished": "^2.4.1",
"parseurl": "^1.3.3",
"statuses": "^2.0.1"
},
"engines": {
"node": ">= 18.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/fresh": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/fresh/-/fresh-2.0.0.tgz",
"integrity": "sha512-Rx/WycZ60HOaqLKAi6cHRKKI7zxWbJ31MhntmtwMoaTeF7XFH9hhBp8vITaMidfljRQ6eYWCKkaTK+ykVJHP2A==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/iconv-lite": {
"version": "0.7.2",
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.2.tgz",
"integrity": "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw==",
"license": "MIT",
"dependencies": {
"safer-buffer": ">= 2.1.2 < 3.0.0"
},
"engines": {
"node": ">=0.10.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/media-typer": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/media-typer/-/media-typer-1.1.0.tgz",
"integrity": "sha512-aisnrDP4GNe06UcKFnV5bfMNPBUw4jsLGaWwWfnH3v02GnBuXX2MCVn5RbrWo0j3pczUilYblq7fQ7Nw2t5XKw==",
"license": "MIT",
"engines": {
"node": ">= 0.8"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/merge-descriptors": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-2.0.0.tgz",
"integrity": "sha512-Snk314V5ayFLhp3fkUREub6WtjBfPdCPY1Ln8/8munuLuiYhsABgBVWsozAG+MWMbVEvcdcpbi9R7ww22l9Q3g==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/mime-db": {
"version": "1.54.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.54.0.tgz",
"integrity": "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/mime-types": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-3.0.2.tgz",
"integrity": "sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A==",
"license": "MIT",
"dependencies": {
"mime-db": "^1.54.0"
},
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT"
},
"node_modules/@modelcontextprotocol/sdk/node_modules/negotiator": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-1.0.0.tgz",
"integrity": "sha512-8Ofs/AUQh8MaEcrlq5xOX0CQ9ypTF5dl78mjlMNfOK08fzpgTHQRQPBxcPlEtIw0yRpws+Zo/3r+5WRby7u3Gg==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/raw-body": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/raw-body/-/raw-body-3.0.2.tgz",
"integrity": "sha512-K5zQjDllxWkf7Z5xJdV0/B0WTNqx6vxG70zJE4N0kBs4LovmEYWJzQGxC9bS9RAKu3bgM40lrd5zoLJ12MQ5BA==",
"license": "MIT",
"dependencies": {
"bytes": "~3.1.2",
"http-errors": "~2.0.1",
"iconv-lite": "~0.7.0",
"unpipe": "~1.0.0"
},
"engines": {
"node": ">= 0.10"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/send": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/send/-/send-1.2.1.tgz",
"integrity": "sha512-1gnZf7DFcoIcajTjTwjwuDjzuz4PPcY2StKPlsGAQ1+YH20IRVrBaXSWmdjowTJ6u8Rc01PoYOGHXfP1mYcZNQ==",
"license": "MIT",
"dependencies": {
"debug": "^4.4.3",
"encodeurl": "^2.0.0",
"escape-html": "^1.0.3",
"etag": "^1.8.1",
"fresh": "^2.0.0",
"http-errors": "^2.0.1",
"mime-types": "^3.0.2",
"ms": "^2.1.3",
"on-finished": "^2.4.1",
"range-parser": "^1.2.1",
"statuses": "^2.0.2"
},
"engines": {
"node": ">= 18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/serve-static": {
"version": "2.2.1",
"resolved": "https://registry.npmjs.org/serve-static/-/serve-static-2.2.1.tgz",
"integrity": "sha512-xRXBn0pPqQTVQiC8wyQrKs2MOlX24zQ0POGaj0kultvoOCstBQM5yvOhAVSUwOMjQtTvsPWoNCHfPGwaaQJhTw==",
"license": "MIT",
"dependencies": {
"encodeurl": "^2.0.0",
"escape-html": "^1.0.3",
"parseurl": "^1.3.3",
"send": "^1.2.0"
},
"engines": {
"node": ">= 18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/type-is": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-2.1.0.tgz",
"integrity": "sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==",
"license": "MIT",
"dependencies": {
"content-type": "^2.0.0",
"media-typer": "^1.1.0",
"mime-types": "^3.0.0"
},
"engines": {
"node": ">= 18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@modelcontextprotocol/sdk/node_modules/type-is/node_modules/content-type": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
"integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/@types/body-parser": { "node_modules/@types/body-parser": {
"version": "1.19.6", "version": "1.19.6",
"resolved": "https://registry.npmjs.org/@types/body-parser/-/body-parser-1.19.6.tgz", "resolved": "https://registry.npmjs.org/@types/body-parser/-/body-parser-1.19.6.tgz",
@@ -1035,39 +652,6 @@
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/ajv": {
"version": "8.20.0",
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz",
"integrity": "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA==",
"license": "MIT",
"dependencies": {
"fast-deep-equal": "^3.1.3",
"fast-uri": "^3.0.1",
"json-schema-traverse": "^1.0.0",
"require-from-string": "^2.0.2"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/epoberezkin"
}
},
"node_modules/ajv-formats": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/ajv-formats/-/ajv-formats-3.0.1.tgz",
"integrity": "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ==",
"license": "MIT",
"dependencies": {
"ajv": "^8.0.0"
},
"peerDependencies": {
"ajv": "^8.0.0"
},
"peerDependenciesMeta": {
"ajv": {
"optional": true
}
}
},
"node_modules/append-field": { "node_modules/append-field": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/append-field/-/append-field-1.0.0.tgz", "resolved": "https://registry.npmjs.org/append-field/-/append-field-1.0.0.tgz",
@@ -1263,20 +847,6 @@
"url": "https://opencollective.com/express" "url": "https://opencollective.com/express"
} }
}, },
"node_modules/cross-spawn": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz",
"integrity": "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==",
"license": "MIT",
"dependencies": {
"path-key": "^3.1.0",
"shebang-command": "^2.0.0",
"which": "^2.0.1"
},
"engines": {
"node": ">= 8"
}
},
"node_modules/debug": { "node_modules/debug": {
"version": "2.6.9", "version": "2.6.9",
"resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz", "resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
@@ -1457,27 +1027,6 @@
"node": ">= 0.6" "node": ">= 0.6"
} }
}, },
"node_modules/eventsource": {
"version": "3.0.7",
"resolved": "https://registry.npmjs.org/eventsource/-/eventsource-3.0.7.tgz",
"integrity": "sha512-CRT1WTyuQoD771GW56XEZFQ/ZoSfWid1alKGDYMmkt2yl8UXrVR4pspqWNEcqKvVIzg6PAltWjxcSSPrboA4iA==",
"license": "MIT",
"dependencies": {
"eventsource-parser": "^3.0.1"
},
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/eventsource-parser": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/eventsource-parser/-/eventsource-parser-3.1.0.tgz",
"integrity": "sha512-kJezFj9YFAMLeORyi7aCLxLbD5/qWMQnoMVlVPyHIll7lgRJCc3JVln9Vgl9nwQi0YkMnhdGTMNn7CkRRAptMg==",
"license": "MIT",
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/express": { "node_modules/express": {
"version": "4.22.2", "version": "4.22.2",
"resolved": "https://registry.npmjs.org/express/-/express-4.22.2.tgz", "resolved": "https://registry.npmjs.org/express/-/express-4.22.2.tgz",
@@ -1524,46 +1073,6 @@
"url": "https://opencollective.com/express" "url": "https://opencollective.com/express"
} }
}, },
"node_modules/express-rate-limit": {
"version": "8.5.2",
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-8.5.2.tgz",
"integrity": "sha512-5Kb34ipNX694DH48vN9irak1Qx30nb0PLYHXfJgw4YEjiC3ZEmZJhwOp+VfiCYwFzvFTdB9QkArYS5kXa2cx2A==",
"license": "MIT",
"dependencies": {
"ip-address": "^10.2.0"
},
"engines": {
"node": ">= 16"
},
"funding": {
"url": "https://github.com/sponsors/express-rate-limit"
},
"peerDependencies": {
"express": ">= 4.11"
}
},
"node_modules/fast-deep-equal": {
"version": "3.1.3",
"resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz",
"integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==",
"license": "MIT"
},
"node_modules/fast-uri": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.2.tgz",
"integrity": "sha512-rVjf7ArG3LTk+FS6Yw81V1DLuZl1bRbNrev6Tmd/9RaroeeRRJhAt7jg/6YFxbvAQXUCavSoZhPPj6oOx+5KjQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fastify"
},
{
"type": "opencollective",
"url": "https://opencollective.com/fastify"
}
],
"license": "BSD-3-Clause"
},
"node_modules/finalhandler": { "node_modules/finalhandler": {
"version": "1.3.2", "version": "1.3.2",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz", "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz",
@@ -1748,15 +1257,6 @@
"node": ">= 0.4" "node": ">= 0.4"
} }
}, },
"node_modules/hono": {
"version": "4.12.26",
"resolved": "https://registry.npmjs.org/hono/-/hono-4.12.26.tgz",
"integrity": "sha512-uyZtpnYxM9CmQ7QsQknM4zN8EftNqhON1qYeIKM0Se67CCEe2c44xyGURwB0axX2fBDu1dqHrHAc1hmNT8ITkw==",
"license": "MIT",
"engines": {
"node": ">=16.9.0"
}
},
"node_modules/http-errors": { "node_modules/http-errors": {
"version": "2.0.1", "version": "2.0.1",
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz", "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz",
@@ -1831,15 +1331,6 @@
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC" "license": "ISC"
}, },
"node_modules/ip-address": {
"version": "10.2.0",
"resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.2.0.tgz",
"integrity": "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA==",
"license": "MIT",
"engines": {
"node": ">= 12"
}
},
"node_modules/ipaddr.js": { "node_modules/ipaddr.js": {
"version": "1.9.1", "version": "1.9.1",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz", "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@@ -1849,45 +1340,12 @@
"node": ">= 0.10" "node": ">= 0.10"
} }
}, },
"node_modules/is-promise": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/is-promise/-/is-promise-4.0.0.tgz",
"integrity": "sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ==",
"license": "MIT"
},
"node_modules/isarray": { "node_modules/isarray": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz", "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz",
"integrity": "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ==", "integrity": "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/isexe": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz",
"integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==",
"license": "ISC"
},
"node_modules/jose": {
"version": "6.2.3",
"resolved": "https://registry.npmjs.org/jose/-/jose-6.2.3.tgz",
"integrity": "sha512-YYVDInQKFJfR/xa3ojUTl8c2KoTwiL1R5Wg9YCydwH0x0B9grbzlg5HC7mMjCtUJjbQ/YnGEZIhI5tCgfTb4Hw==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/panva"
}
},
"node_modules/json-schema-traverse": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz",
"integrity": "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==",
"license": "MIT"
},
"node_modules/json-schema-typed": {
"version": "8.0.2",
"resolved": "https://registry.npmjs.org/json-schema-typed/-/json-schema-typed-8.0.2.tgz",
"integrity": "sha512-fQhoXdcvc3V28x7C7BMs4P5+kNlgUURe2jmUT1T//oBRMDrqy1QPelJimwZGo7Hg9VPV3EQV5Bnq4hbFy2vetA==",
"license": "BSD-2-Clause"
},
"node_modules/math-intrinsics": { "node_modules/math-intrinsics": {
"version": "1.1.0", "version": "1.1.0",
"resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz", "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
@@ -2045,15 +1503,6 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/once": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
"license": "ISC",
"dependencies": {
"wrappy": "1"
}
},
"node_modules/openai": { "node_modules/openai": {
"version": "6.44.0", "version": "6.44.0",
"resolved": "https://registry.npmjs.org/openai/-/openai-6.44.0.tgz", "resolved": "https://registry.npmjs.org/openai/-/openai-6.44.0.tgz",
@@ -2081,30 +1530,12 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/path-key": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz",
"integrity": "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/path-to-regexp": { "node_modules/path-to-regexp": {
"version": "0.1.13", "version": "0.1.13",
"resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.13.tgz", "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.13.tgz",
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==", "integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/pkce-challenge": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/pkce-challenge/-/pkce-challenge-5.0.1.tgz",
"integrity": "sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ==",
"license": "MIT",
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/process-nextick-args": { "node_modules/process-nextick-args": {
"version": "2.0.1", "version": "2.0.1",
"resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.1.tgz", "resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.1.tgz",
@@ -2193,64 +1624,6 @@
"integrity": "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g==", "integrity": "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/require-from-string": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
"integrity": "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/router": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/router/-/router-2.2.0.tgz",
"integrity": "sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ==",
"license": "MIT",
"dependencies": {
"debug": "^4.4.0",
"depd": "^2.0.0",
"is-promise": "^4.0.0",
"parseurl": "^1.3.3",
"path-to-regexp": "^8.0.0"
},
"engines": {
"node": ">= 18"
}
},
"node_modules/router/node_modules/debug": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
"integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/router/node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT"
},
"node_modules/router/node_modules/path-to-regexp": {
"version": "8.4.2",
"resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-8.4.2.tgz",
"integrity": "sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==",
"license": "MIT",
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/safe-buffer": { "node_modules/safe-buffer": {
"version": "5.2.1", "version": "5.2.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
@@ -2328,27 +1701,6 @@
"integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==", "integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==",
"license": "ISC" "license": "ISC"
}, },
"node_modules/shebang-command": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz",
"integrity": "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==",
"license": "MIT",
"dependencies": {
"shebang-regex": "^3.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/shebang-regex": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/shebang-regex/-/shebang-regex-3.0.0.tgz",
"integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/side-channel": { "node_modules/side-channel": {
"version": "1.1.1", "version": "1.1.1",
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.1.tgz", "resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.1.tgz",
@@ -2554,27 +1906,6 @@
"node": ">= 0.8" "node": ">= 0.8"
} }
}, },
"node_modules/which": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz",
"integrity": "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA==",
"license": "ISC",
"dependencies": {
"isexe": "^2.0.0"
},
"bin": {
"node-which": "bin/node-which"
},
"engines": {
"node": ">= 8"
}
},
"node_modules/wrappy": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
},
"node_modules/xtend": { "node_modules/xtend": {
"version": "4.0.2", "version": "4.0.2",
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz", "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
@@ -2583,24 +1914,6 @@
"engines": { "engines": {
"node": ">=0.4" "node": ">=0.4"
} }
},
"node_modules/zod": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz",
"integrity": "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
},
"node_modules/zod-to-json-schema": {
"version": "3.25.2",
"resolved": "https://registry.npmjs.org/zod-to-json-schema/-/zod-to-json-schema-3.25.2.tgz",
"integrity": "sha512-O/PgfnpT1xKSDeQYSCfRI5Gy3hPf91mKVDuYLUHZJMiDFptvP41MSnWofm8dnCm0256ZNfZIM7DSzuSMAFnjHA==",
"license": "ISC",
"peerDependencies": {
"zod": "^3.25.28 || ^4"
}
} }
} }
} }

1
backend/package.json
View File

@@ -10,7 +10,6 @@
"start": "node dist/index.js" "start": "node dist/index.js"
}, },
"dependencies": { "dependencies": {
"@modelcontextprotocol/sdk": "^1.29.0",
"axios": "^1.7.0", "axios": "^1.7.0",
"cors": "^2.8.5", "cors": "^2.8.5",
"dotenv": "^16.4.5", "dotenv": "^16.4.5",

BIN
backend/piper/ca_ES-upc_ona-medium.onnx
View File

Binary file not shown.

493
backend/piper/ca_ES-upc_ona-medium.onnx.json
View File

@@ -1,493 +0,0 @@
{
"audio": {
"sample_rate": 22050,
"quality": "medium"
},
"espeak": {
"voice": "ca"
},
"inference": {
"noise_scale": 0.667,
"length_scale": 1,
"noise_w": 0.8
},
"phoneme_type": "espeak",
"phoneme_map": {},
"phoneme_id_map": {
"_": [
0
],
"^": [
1
],
"$": [
2
],
" ": [
3
],
"!": [
4
],
"'": [
5
],
"(": [
6
],
")": [
7
],
",": [
8
],
"-": [
9
],
".": [
10
],
":": [
11
],
";": [
12
],
"?": [
13
],
"a": [
14
],
"b": [
15
],
"c": [
16
],
"d": [
17
],
"e": [
18
],
"f": [
19
],
"h": [
20
],
"i": [
21
],
"j": [
22
],
"k": [
23
],
"l": [
24
],
"m": [
25
],
"n": [
26
],
"o": [
27
],
"p": [
28
],
"q": [
29
],
"r": [
30
],
"s": [
31
],
"t": [
32
],
"u": [
33
],
"v": [
34
],
"w": [
35
],
"x": [
36
],
"y": [
37
],
"z": [
38
],
"æ": [
39
],
"ç": [
40
],
"ð": [
41
],
"ø": [
42
],
"ħ": [
43
],
"ŋ": [
44
],
"œ": [
45
],
"ǀ": [
46
],
"ǁ": [
47
],
"ǂ": [
48
],
"ǃ": [
49
],
"ɐ": [
50
],
"ɑ": [
51
],
"ɒ": [
52
],
"ɓ": [
53
],
"ɔ": [
54
],
"ɕ": [
55
],
"ɖ": [
56
],
"ɗ": [
57
],
"ɘ": [
58
],
"ə": [
59
],
"ɚ": [
60
],
"ɛ": [
61
],
"ɜ": [
62
],
"ɞ": [
63
],
"ɟ": [
64
],
"ɠ": [
65
],
"ɡ": [
66
],
"ɢ": [
67
],
"ɣ": [
68
],
"ɤ": [
69
],
"ɥ": [
70
],
"ɦ": [
71
],
"ɧ": [
72
],
"ɨ": [
73
],
"ɪ": [
74
],
"ɫ": [
75
],
"ɬ": [
76
],
"ɭ": [
77
],
"ɮ": [
78
],
"ɯ": [
79
],
"ɰ": [
80
],
"ɱ": [
81
],
"ɲ": [
82
],
"ɳ": [
83
],
"ɴ": [
84
],
"ɵ": [
85
],
"ɶ": [
86
],
"ɸ": [
87
],
"ɹ": [
88
],
"ɺ": [
89
],
"ɻ": [
90
],
"ɽ": [
91
],
"ɾ": [
92
],
"ʀ": [
93
],
"ʁ": [
94
],
"ʂ": [
95
],
"ʃ": [
96
],
"ʄ": [
97
],
"ʈ": [
98
],
"ʉ": [
99
],
"ʊ": [
100
],
"ʋ": [
101
],
"ʌ": [
102
],
"ʍ": [
103
],
"ʎ": [
104
],
"ʏ": [
105
],
"ʐ": [
106
],
"ʑ": [
107
],
"ʒ": [
108
],
"ʔ": [
109
],
"ʕ": [
110
],
"ʘ": [
111
],
"ʙ": [
112
],
"ʛ": [
113
],
"ʜ": [
114
],
"ʝ": [
115
],
"ʟ": [
116
],
"ʡ": [
117
],
"ʢ": [
118
],
"ʲ": [
119
],
"ˈ": [
120
],
"ˌ": [
121
],
"ː": [
122
],
"ˑ": [
123
],
"˞": [
124
],
"β": [
125
],
"θ": [
126
],
"χ": [
127
],
"ᵻ": [
128
],
"ⱱ": [
129
],
"0": [
130
],
"1": [
131
],
"2": [
132
],
"3": [
133
],
"4": [
134
],
"5": [
135
],
"6": [
136
],
"7": [
137
],
"8": [
138
],
"9": [
139
],
"̧": [
140
],
"̃": [
141
],
"̪": [
142
],
"̯": [
143
],
"̩": [
144
],
"ʰ": [
145
],
"ˤ": [
146
],
"ε": [
147
],
"↓": [
148
],
"#": [
149
],
"\"": [
150
],
"↑": [
151
],
"̺": [
152
],
"̻": [
153
]
},
"num_symbols": 256,
"num_speakers": 1,
"speaker_id_map": {},
"piper_version": "1.0.0",
"language": {
"code": "ca_ES",
"family": "ca",
"region": "ES",
"name_native": "Català",
"name_english": "Catalan",
"country_english": "Spain"
},
"dataset": "upc_ona"
}

5
backend/prompts/preamble.md
View File

@@ -1,5 +0,0 @@
Ets la QuiBot, un robot femení que ajuda als nens a aprendre sobre quimica. Disposes de dos rodes i dos braços.
Has de ser educada i tenir perspectiva de gènere.
Les teves respostes han de ser curtes.

1
backend/quibot-audio-1781783002989.txt Normal file
View File

@@ -0,0 +1 @@
Col·la, pítalo, la ola, ola.

1
backend/quibot-audio-1781783032108.txt Normal file
View File

@@ -0,0 +1 @@
Hola, què tal, hola, hola, hola, hola...

1
backend/quibot-audio-1781783047628.txt Normal file
View File

@@ -0,0 +1 @@
Hola, que tal, bon dia.

10
backend/run-mcp.sh
View File

@@ -1,10 +0,0 @@
#!/bin/bash
REMOTE_PORT=2223
HOST=ollama.epsem.aranroig.com
PORT=5001
REMOTE_USER=root
REMOTE_HOST=ollama.epsem.aranroig.com
ssh -p ${REMOTE_PORT} -N -R ${HOST}:${PORT}:localhost:${PORT} -o ServerAliveInterval=30 \
-o ServerAliveCountMax=3 \
-o ExitOnForwardFailure=yes \
${REMOTE_USER}@${REMOTE_HOST}

22
backend/src/config.ts
View File

@@ -1,6 +1,4 @@
import dotenv from 'dotenv'; import dotenv from 'dotenv';
import { readFileSync } from 'fs';
import { join } from 'path';
dotenv.config(); dotenv.config();
@@ -8,14 +6,6 @@ let _raspberryHost = process.env.RASPBERRY_PI_HOST ?? 'http://raspberrypi.local'
let _raspberryPort = Number(process.env.RASPBERRY_PI_PORT) || 8000; let _raspberryPort = Number(process.env.RASPBERRY_PI_PORT) || 8000;
let _token = process.env.QUIBOT_TOKEN ?? 'MY_SECRET_TOKEN'; let _token = process.env.QUIBOT_TOKEN ?? 'MY_SECRET_TOKEN';
const APP_PORT = Number(process.env.PORT) || 5000; const APP_PORT = Number(process.env.PORT) || 5000;
const piperUrl = process.env.PIPER_URL ?? '';
const mcpUrl = process.env.MCP_URL ?? '';
const llamacppUrl = process.env.LLAMA_CPP_URL ?? '';
const llamacppApiKey = process.env.LLAMA_API_KEY ?? '';
const llamaPreambleRaw = process.env.LLAMA_PREAMBLE ?? '';
const llamacppPreamble = llamaPreambleRaw.endsWith('.md')
? readFileSync(llamaPreambleRaw, 'utf-8')
: llamaPreambleRaw;
export const getRaspberryHost = () => _raspberryHost; export const getRaspberryHost = () => _raspberryHost;
export const getRaspberryPort = () => _raspberryPort; export const getRaspberryPort = () => _raspberryPort;
@@ -41,16 +31,4 @@ export const getConfig = () => ({
token: getToken(), token: getToken(),
}); });
export const getLlamacppUrl = () => llamacppUrl;
export const getLlamacppApiKey = () => llamacppApiKey;
export const getLlamacppPreamble = () => llamacppPreamble;
export const getPiperUrl = () => piperUrl;
export const getPiperModelDir = () =>
process.env.PIPER_MODELS_DIR || join('/tmp', 'quibot-piper-models');
export const getPiperModel = () =>
process.env.PIPER_MODEL ||
join(getPiperModelDir(), 'ca_ES-upc_ona-medium.onnx');
export const getMcpUrl = () => mcpUrl;
export const getAppPort = () => APP_PORT; export const getAppPort = () => APP_PORT;

47
backend/src/controllers/audio.controller.ts
View File

@@ -1,14 +1,15 @@
import { Router } from 'express'; import { Router } from 'express';
import multer from 'multer'; import multer from 'multer';
import { join } from 'path'; import { execFile } from 'child_process';
import { tmpdir } from 'os'; import { tmpdir } from 'os';
import { rm, writeFile } from 'fs'; import { join } from 'path';
import { promisify } from 'util'; import { promisify } from 'util';
import { whisperService } from '../services/whisper.service.js'; import { writeFile, unlink } from 'fs';
import { raspiService } from '../services/raspi.service.js'; import { raspiService } from '../services/raspi.service.js';
import { llamacppService } from '../services/llama.service.js';
const unlinkAsync = promisify(rm); const execFileAsync = promisify(execFile);
const writeFileAsync = promisify(writeFile); const writeFileAsync = promisify(writeFile);
const unlinkAsync = promisify(unlink);
const router = Router(); const router = Router();
@@ -68,9 +69,11 @@ router.post('/process/:filename', async (req, res) => {
} }
}); });
const whisperModel = process.env.WHISPER_MODEL ?? 'base';
const whisperLanguage = process.env.WHISPER_LANGUAGE ?? 'ca';
router.post('/upload', upload.single('file'), async (req, res) => { router.post('/upload', upload.single('file'), async (req, res) => {
let tmpFile: string | undefined; let tmpFile: string | undefined;
let tmpTxt: string | undefined;
try { try {
if (!req.file) { if (!req.file) {
return res.status(400).json({ error: 'No audio file provided' }); return res.status(400).json({ error: 'No audio file provided' });
@@ -80,24 +83,23 @@ router.post('/upload', upload.single('file'), async (req, res) => {
tmpFile = join(tmpdir(), `quibot-audio-${Date.now()}.${ext}`); tmpFile = join(tmpdir(), `quibot-audio-${Date.now()}.${ext}`);
await writeFileAsync(tmpFile, req.file.buffer); await writeFileAsync(tmpFile, req.file.buffer);
const transcription = await whisperService.transcribe(tmpFile); console.log(`[whisper] Model: ${whisperModel}, Language: ${whisperLanguage}, File: ${tmpFile}`);
console.log(transcription);
const txtPath = join(tmpdir(), `quibot-audio-${Date.now()}.txt`); const { stdout, stderr } = await execFileAsync('whisper', [
tmpTxt = txtPath; tmpFile,
await writeFileAsync(txtPath, transcription); '--model', whisperModel,
'--language', whisperLanguage,
'--output_format', 'txt',
], { maxBuffer: 50 * 1024 * 1024 });
const llmResponse = await llamacppService.chatWithMcpTools(transcription).catch( if (stderr) {
(err: unknown) => { console.log(`[whisper] stderr: ${stderr}`);
const msg = err instanceof Error ? err.message : String(err); }
console.error(`[audio] llama.cpp failed: ${msg}`);
return undefined; const transcription = stdout.trim();
},
);
res.json({ res.json({
transcription, transcription,
llmResponse,
originalFilename: req.file.originalname, originalFilename: req.file.originalname,
}); });
} catch (err: unknown) { } catch (err: unknown) {
@@ -111,13 +113,6 @@ router.post('/upload', upload.single('file'), async (req, res) => {
// ignore cleanup errors // ignore cleanup errors
} }
} }
if (tmpTxt) {
try {
await unlinkAsync(tmpTxt);
} catch {
// ignore cleanup errors
}
}
} }
}); });

101
backend/src/controllers/text-command.controller.ts Normal file
View File

@@ -0,0 +1,101 @@
import { Router } from 'express';
import { raspiService } from '../services/raspi.service.js';
const router = Router();
const commandMap = [
// Catalan forward
{ words: ['endavant', 'avança'], action: 'forward' },
// Catalan backward
{ words: ['atras', 'enrere', 'atras'], action: 'backward' },
// Catalan left
{ words: ['esquerra', 'esquerre', 'stoppa'], action: 'left' },
// Catalan right
{ words: ['dreta', 'destre'], action: 'right' },
// Stop
{ words: ['atura', 'atura', 'pare', 'stop', 'parada', 'aturar'], action: 'stop' },
// Pick up / grab
{ words: ['recull', 'pega', 'pilla', 'agafa'], action: 'pick' },
// Eject / throw
{ words: ['llança', 'tira', 'expulsa', ' llença'], action: 'eject' },
// Spanish forward
{ words: ['adelante'], action: 'forward' },
// Spanish backward
{ words: ['atras', 'atrás', 'reversa', 'al tras'], action: 'backward' },
// Spanish left
{ words: ['izquierda', 'izq'], action: 'left' },
// Spanish right
{ words: ['derecha', 'der'], action: 'right' },
// Spanish stop
{ words: ['para', 'stop', 'pare', 'deten', 'frena', 'alto'], action: 'stop' },
];
function resolveCommand(text: string) {
const lower = text.toLowerCase().normalize("NFD").replace(/[\u0300-\u036f]/g, "");
for (const entry of commandMap) {
for (const word of entry.words) {
const normalizedWord = word.normalize("NFD").replace(/[\u0300-\u036f]/g, "");
if (lower.includes(normalizedWord)) {
return entry.action;
}
}
}
return null;
}
router.post('/text', async (req, res) => {
const { text } = req.body;
if (!text || typeof text !== 'string') {
return res.status(400).json({ error: 'Text is required' });
}
const action = resolveCommand(text);
if (!action) {
return res.json({ recognized: text.trim(), action: null, message: 'No motor command matched. Use: endavant, atras, esquerra, dreta, atura, recull, llança' });
}
try {
switch (action) {
case 'forward': {
const result = await raspiService.motorStepForward();
return res.json({ recognized: text.trim(), action: 'forward', result });
}
case 'backward': {
const result = await raspiService.motorStepBackward();
return res.json({ recognized: text.trim(), action: 'backward', result });
}
case 'left': {
await raspiService.motorStepForward();
await new Promise(r => setTimeout(r, 600));
await raspiService.motorStop();
return res.json({ recognized: text.trim(), action: 'turn-left', result: { status: 'turned left' } });
}
case 'right': {
await raspiService.motorStepBackward();
await new Promise(r => setTimeout(r, 600));
await raspiService.motorStop();
return res.json({ recognized: text.trim(), action: 'turn-right', result: { status: 'turned right' } });
}
case 'stop': {
const result = await raspiService.motorStop();
return res.json({ recognized: text.trim(), action: 'stop', result });
}
case 'pick': {
return res.json({ recognized: text.trim(), action: 'pick', message: 'Pick command received' });
}
case 'eject': {
return res.json({ recognized: text.trim(), action: 'eject', message: 'Eject command received' });
}
}
} catch (err: unknown) {
const message = err instanceof Error ? err.message : 'Unknown error';
return res.status(500).json({
recognized: text.trim(),
action,
error: `Command execution failed: ${message}`,
});
}
});
export default router;

52
backend/src/controllers/tts.controller.ts
View File

@@ -1,52 +0,0 @@
import { Router } from 'express';
import { randomUUID } from 'crypto';
import { join } from 'path';
import { mkdirSync, writeFileSync } from 'fs';
import { piperService } from '../services/piper.service.js';
import { getToken } from '../config.js';
const router = Router();
const TTS_AUDIO_DIR = join('/tmp', 'quibot-audio', 'tts');
try { mkdirSync(TTS_AUDIO_DIR, { recursive: true }); } catch { /* ignore */ }
router.post('/', async (req, res) => {
try {
const text = (req.query.text as string) || (req.body?.text as string);
// Accept 'lang' or 'language' — APK sends 'language', old tests use 'lang'
const lang = (req.query.lang as string) || (req.query.language as string) || 'ca';
const token = (req.query.token as string) || '';
if (!text?.trim()) {
return res.status(400).json({ error: 'Missing query parameter: text' });
}
const expectedToken = getToken();
if (token && token !== expectedToken) {
return res.status(401).json({ error: 'Unauthorized: invalid token' });
}
console.log(`[tts] Generating audio for text (${lang}): "${text.substring(0, 60)}..."`);
// Ensure Piper subprocess is initialized before synthesis
await piperService.initWav();
const wavBuffer = await piperService.synthWav(text.trim());
const filename = `${randomUUID()}.wav`;
writeFileSync(join(TTS_AUDIO_DIR, filename), wavBuffer);
console.log(`[tts] Audio saved: ${filename}`);
return res.json({
audioUrl: `/tts-audio/${filename}`,
filename,
});
} catch (err: unknown) {
const message = err instanceof Error ? err.message : 'Unknown error';
console.error(`[tts] Error: ${message}`);
return res.status(500).json({ error: `TTS synthesis failed: ${message}` });
}
});
export default router;

24
backend/src/index.ts
View File

@@ -2,9 +2,6 @@ import express from 'express';
import cors from 'cors'; import cors from 'cors';
import router from './routes/router.js'; import router from './routes/router.js';
import { getAppPort, getConfig } from './config.js'; import { getAppPort, getConfig } from './config.js';
import { whisperService } from './services/whisper.service.js';
import { piperService as piperWorker } from './services/piper.service.js';
import { mcpClient } from './services/mcp.service.js';
const app = express(); const app = express();
@@ -16,9 +13,6 @@ app.use('/audio', express.json());
app.use('/motor', express.json()); app.use('/motor', express.json());
app.use('/commands', express.json()); app.use('/commands', express.json());
// Serve generated TTS audio files to the APK
app.use('/tts-audio', express.static('/tmp/quibot-audio/tts'));
app.use(router); app.use(router);
app.get('/health', (_req, res) => { app.get('/health', (_req, res) => {
@@ -26,22 +20,6 @@ app.get('/health', (_req, res) => {
res.json({ status: 'ok', settings }); res.json({ status: 'ok', settings });
}); });
const server = app.listen(getAppPort(), async () => { app.listen(getAppPort(), () => {
console.log(`QuiBot backend listening on port ${getAppPort()}`); console.log(`QuiBot backend listening on port ${getAppPort()}`);
whisperService.spawn();
piperWorker.initWav().catch(() => { /* model may not exist yet → lazy init on first TTS call */ });
mcpClient.connect().catch((err) => {
console.error(`[mcp] Failed to start MCP client: ${err instanceof Error ? err.message : String(err)}`);
});
}); });
async function shutdown(signal: string) {
console.log(`[server] ${signal} received, shutting down...`);
server.close(async () => {
await Promise.all([whisperService.shutdown(), piperWorker.shutdown(), mcpClient.shutdown()]);
process.exit(0);
});
}
process.on('SIGINT', () => shutdown('SIGINT'));
process.on('SIGTERM', () => shutdown('SIGTERM'));

120
backend/src/piper-worker.py
View File

@@ -1,120 +0,0 @@
#!/usr/bin/env python3
"""Persistent Piper TTS worker – single subprocess, model loaded once."""
import os
import sys
import json
import wave
import io
class PiperStdoutSink:
"""Wraps stdout so we write exactly N bytes."""
def __init__(self):
self._remaining = 0
def set_length(self, length: int):
self._remaining = length
def write(self, data: bytes):
to_write = min(len(data), self._remaining)
if to_write > 0:
sys.stdout.buffer.write(data[:to_write])
sys.stdout.buffer.flush()
self._remaining -= to_write
def main():
from piper import PiperVoice
model_path = ""
config_path = None
voice = None
# Defaults (will be overridden by init message)
_default_dir = os.environ.get("PIPER_MODELS_DIR", "/tmp/quibot-piper-models")
DEFAULT_MODEL = os.path.join(_default_dir, "ca_ES-upc_ona-medium.onnx")
DEFAULT_CONFIG = os.path.join(_default_dir, "ca_ES-upc_ona-medium.onnx.json")
# Signal node that the process is alive and listening
print(json.dumps({"type": "ready"}), flush=True)
for line in sys.stdin:
line = line.strip()
if not line:
continue
try:
msg = json.loads(line)
except json.JSONDecodeError:
continue
if msg.get("type") == "init":
model_path = msg.get("model", DEFAULT_MODEL) or DEFAULT_MODEL
config_path = msg.get("config", None)
print(f"[piper-worker] Loading model='{model_path}'", file=sys.stderr, flush=True)
try:
voice = PiperVoice.load(model_path, config_path=config_path)
print(json.dumps({"type": "init_ok"}), flush=True)
except Exception as exc:
err_msg = str(exc).replace('"', '\\"')
print(json.dumps({"type": "init_error", "error": err_msg}), flush=True)
elif msg.get("type") == "synthesize":
text = msg.get("text", "")
msg_id = msg.get("msgId", "")
# NEW: output file path from message
out_path = msg.get("outPath")
if not voice:
print(json.dumps({
"type": "error",
"text": "Model not loaded, send init first",
"msgId": msg_id,
}), flush=True)
continue
if not out_path:
print(json.dumps({
"type": "error",
"text": "Missing outPath",
"msgId": msg_id,
}), flush=True)
continue
try:
import io
import wave
buf = io.BytesIO()
wf = wave.open(buf, 'wb')
voice.synthesize_wav(text, wf)
wf.close()
wav_bytes = buf.getvalue()
# Write to file instead of stdout
os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
with open(out_path, "wb") as f:
f.write(wav_bytes)
print(json.dumps({
"type": "synthesized",
"bytes": len(wav_bytes),
"msgId": msg_id,
"outPath": out_path
}), flush=True)
except Exception as exc:
err_msg = str(exc).replace('"', '\\"')
print(json.dumps({
"type": "error",
"text": err_msg,
"msgId": msg_id,
}), flush=True)
if __name__ == "__main__":
main()

4
backend/src/routes/router.ts
View File

@@ -3,14 +3,14 @@ import motorController from '../controllers/motor.controller.js';
import audioController from '../controllers/audio.controller.js'; import audioController from '../controllers/audio.controller.js';
import commandController from '../controllers/command.controller.js'; import commandController from '../controllers/command.controller.js';
import settingsController from '../controllers/settings.controller.js'; import settingsController from '../controllers/settings.controller.js';
import ttsController from '../controllers/tts.controller.js'; import textCommandController from '../controllers/text-command.controller.js';
const router = Router(); const router = Router();
router.use('/motor', motorController); router.use('/motor', motorController);
router.use('/audio', audioController); router.use('/audio', audioController);
router.use('/commands', commandController); router.use('/commands', commandController);
router.use('/commands/text', textCommandController);
router.use('/settings', settingsController); router.use('/settings', settingsController);
router.use('/tts', ttsController);
export default router; export default router;

171
backend/src/services/llama.service.ts
View File

@@ -1,171 +0,0 @@
import { getLlamacppUrl, getLlamacppApiKey, getLlamacppPreamble } from '../config.js';
import { mcpClient, McpToolDef } from './mcp.service.js';
interface LlamaMessage {
role: string;
content?: string | null;
tool_call_id?: string;
tool_calls?: Array<{
id: string;
type: string;
function: { name: string; arguments: string };
}>;
}
interface LlamaToolCallResult {
content: Array<{
type: string;
text?: string;
}>;
isError?: boolean;
}
interface LlamaToolDefinition {
type: 'function';
function: {
name: string;
description: string;
parameters: object;
};
}
interface LlamaRequest {
messages: LlamaMessage[];
tools?: LlamaToolDefinition[];
tool_choice?: 'auto' | 'none';
}
interface LlamaResponseChoice {
message?: {
content?: string;
tool_calls?: Array<{
id: string;
type: string;
function: { name: string; arguments: string };
}>;
};
}
interface LlamaResponse {
choices?: LlamaResponseChoice[];
}
const MAX_TOOL_ITERATIONS = 10;
export const llamacppService = {
async chat(messages: Array<{ role: string; content: string }>): Promise<string> {
let history: LlamaMessage[] = messages.map(m => ({ role: m.role, content: m.content }));
const apiUrl = getLlamacppUrl();
if (!apiUrl) return '';
const apiKey = getLlamacppApiKey();
const headers: Record<string, string> = { 'Content-Type': 'application/json' };
if (apiKey) headers['Authorization'] = `Bearer ${apiKey}`;
const request: LlamaRequest = { messages: history };
const res = await fetch(apiUrl, { method: 'POST', headers, body: JSON.stringify(request) });
if (!res.ok) {
const text = await res.text().catch(() => '');
throw new Error(`llama.cpp request failed (${res.status}): ${text.slice(0, 300)}`);
}
const data = (await res.json()) as LlamaResponse;
const choice = data.choices?.[0];
if (!choice?.message || !choice.message.content) {
return '';
}
return choice.message.content.trim();
},
async chatWithPreamble(userText: string): Promise<string> {
const preamble = getLlamacppPreamble();
const msgs = preamble
? [{ role: 'system' as const, content: preamble }, { role: 'user' as const, content: userText }]
: [{ role: 'user' as const, content: userText }];
return this.chat(msgs);
},
async chatWithMcpTools(userText: string): Promise<string> {
const preamble = getLlamacppPreamble();
const initialMessages: LlamaMessage[] = preamble
? [{ role: 'system', content: preamble }, { role: 'user', content: userText }]
: [{ role: 'user', content: userText }];
if (!mcpClient.getReady()) {
console.log('[llama] MCP not ready, falling back to preamble-only chat');
return this.chatWithPreamble(userText);
}
const tools = mcpClient.getTools();
const llamaTools: LlamaToolDefinition[] = tools.map((t) => ({
type: 'function',
function: {
name: t.name,
description: t.description,
parameters: t.inputSchema,
},
}));
return this._chatWithTools(initialMessages, llamaTools);
},
async _chatWithTools(messages: LlamaMessage[], tools: LlamaToolDefinition[]): Promise<string> {
const apiUrl = getLlamacppUrl();
if (!apiUrl) return '';
let iter = 0;
const history: LlamaMessage[] = messages.map((m) => ({ ...m }));
while (iter < MAX_TOOL_ITERATIONS) {
const apiKey = getLlamacppApiKey();
const headers: Record<string, string> = { 'Content-Type': 'application/json' };
if (apiKey) headers['Authorization'] = `Bearer ${apiKey}`;
const body: LlamaRequest = { messages: history, tools, tool_choice: 'auto' };
const res = await fetch(apiUrl, { method: 'POST', headers, body: JSON.stringify(body) });
if (!res.ok) {
const text = await res.text().catch(() => '');
throw new Error(`llama.cpp request failed (${res.status}): ${text.slice(0, 300)}`);
}
const data = (await res.json()) as LlamaResponse;
const choice = data.choices?.[0];
if (!choice?.message) {
throw new Error('llama.cpp response has no message');
}
if (!choice.message.tool_calls || choice.message.tool_calls.length === 0) {
return choice.message.content?.trim() ?? '';
}
history.push({ role: 'assistant', ...choice.message });
for (const toolCall of choice.message.tool_calls) {
let resultText = '';
try {
const args = JSON.parse(toolCall.function.arguments);
resultText = await mcpClient.callTool(toolCall.function.name, args);
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
resultText = `Error calling tool "${toolCall.function.name}": ${msg}`;
}
history.push({
role: 'tool',
content: resultText,
tool_call_id: toolCall.id,
});
}
iter++;
}
throw new Error(`Exceeded max tool-call iterations (${MAX_TOOL_ITERATIONS})`);
},
};

105
backend/src/services/mcp.service.ts
View File

@@ -1,105 +0,0 @@
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { SSEClientTransport } from '@modelcontextprotocol/sdk/client/sse.js';
import { getMcpUrl } from '../config.js';
export interface McpToolDef {
name: string;
description: string;
inputSchema: object;
}
let mc: Client | null = null;
let cachedTools: McpToolDef[] = [];
let connected = false;
let connecting = false;
async function connectInternal(): Promise<void> {
if (connected) return;
if (connecting) throw new Error('MCP client connection already in progress');
connecting = true;
const rawUrl = getMcpUrl();
if (!rawUrl) {
console.warn('[mcp] MCP_URL not configured, tools disabled');
connecting = false;
return;
}
// Ensure the URL points at the /sse endpoint (FastMCP default)
let connectUrl = rawUrl;
try {
const u = new URL(rawUrl);
if (u.pathname === '/' || u.pathname === '') {
u.pathname = '/sse';
connectUrl = u.toString();
}
} catch {
// not a valid URL, use as-is
}
console.log(`[mcp] Connecting to ${connectUrl}...`);
try {
mc = new Client(
{ name: 'quibot-backend', version: '1.0.0' },
{ capabilities: {} },
);
const transport = new SSEClientTransport(new URL(connectUrl));
await mc.connect(transport);
console.log('[mcp] Connected, listing tools...');
const toolsResult = await mc.listTools();
cachedTools = (toolsResult.tools ?? []).map((t) => ({
name: t.name,
description: t.description ?? '',
inputSchema: t.inputSchema as object,
}));
connected = true;
connecting = false;
console.log(`[mcp] Connected to MCP server with ${cachedTools.length} tool(s): ${cachedTools.map((t) => t.name).join(', ') || '(none)'}`);
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
connecting = false;
console.error(`[mcp] Connection failed: ${msg}`);
throw err;
}
}
export const mcpClient = {
async connect(): Promise<void> {
await connectInternal();
},
getReady(): boolean {
return connected;
},
getTools(): readonly McpToolDef[] {
return cachedTools;
},
async callTool(name: string, args: Record<string, unknown>): Promise<string> {
if (!mc) throw new Error('MCP client not connected');
const result = await mc.callTool({ name, arguments: args });
const content = result.content as Array<{ type: string; text?: string }>;
const texts = content
.filter((c) => c.type === 'text')
.map((c) => c.text ?? '');
return texts.join('\n') || '[MCP tool returned no text content]';
},
async shutdown(): Promise<void> {
if (mc) {
try {
await mc.close();
} catch {
// ignore close errors on shutdown
}
mc = null;
}
connected = false;
cachedTools = [];
},
};

26
backend/src/services/piper.http.service.ts
View File

@@ -1,26 +0,0 @@
import axios from 'axios';
import { getPiperUrl } from '../config.js';
export interface PiperSynthesisParams {
text: string;
lang?: string;
}
class PiperHttpService {
async synthesize(params: PiperSynthesisParams): Promise<Buffer> {
const piperUrl = getPiperUrl();
if (!piperUrl) throw new Error('PIPER_URL not configured');
const speakerId = params.lang === 'en' ? 1 : 0;
const response = await axios.post(
`${piperUrl}/api/tts`,
{ text: params.text, speaker_id: speakerId },
{ responseType: 'arraybuffer', timeout: 30_000 },
);
return Buffer.from(response.data, 'binary');
}
}
export const piperService = new PiperHttpService();

285
backend/src/services/piper.service.ts
View File

@@ -1,285 +0,0 @@
import { spawn, ChildProcess } from 'child_process';
import { join } from 'path';
import { fileURLToPath } from 'url';
import { randomUUID } from 'crypto';
import { readFileSync, rmSync, readdirSync, statSync, unlinkSync, mkdirSync } from 'fs';
import { piperService as httpPiperService } from './piper.http.service.js';
import { getPiperUrl, getPiperModel } from '../config.js';
const __filename = fileURLToPath(import.meta.url);
const __dirname = join(__filename, '..');
const SCRIPT_DIR = join(__dirname, '..');
const TTS_DIR = join('/tmp', 'quibot-audio', 'tts-piper');
mkdirSync(TTS_DIR, { recursive: true });
// ─── type-guard for JSON messages from piper-worker ───
type PiperMsg =
| { type: 'ready' }
| { type: 'init_ok' }
| { type: 'init_error'; error: string }
| { type: 'synthesized'; outPath: string; bytes?: number; msgId: string }
| { type: 'error'; text: string; msgId: string };
class PiperLocalService {
private proc: ChildProcess | null = null;
private pendingInit: Promise<void> | null = null;
private cleanupTimer: ReturnType<typeof setInterval> | null = null;
// ── spawn + stdout parser (simple: pure JSON on stdout, WAV on disk) ──
private setupStdout(): void {
if (!this.proc?.stdout) return;
let buf = '';
this.proc.stdout.on('data', (chunk: Buffer) => {
buf += chunk.toString();
while (true) {
const nl = buf.indexOf('\n');
if (nl === -1) break;
const line = buf.slice(0, nl).trim();
buf = buf.slice(nl + 1);
if (!line) continue;
this.handleLine(line);
}
});
}
private handleLine(line: string): void {
console.log('[RX]', line);
let msg: PiperMsg;
try {
msg = JSON.parse(line) as PiperMsg;
} catch {
console.warn('[piper-svc] Invalid JSON:', line);
return;
}
switch (msg.type) {
case 'ready':
break;
case 'init_ok':
this.resolveInit();
break;
case 'init_error':
this.resolveInitError(new Error(msg.error));
break;
case 'synthesized':
this.resolveResponse(msg.msgId, msg.outPath);
break;
case 'error':
this.rejectResponse(
msg.msgId,
new Error(msg.text)
);
break;
}
}
private async writeStdin(line: string): Promise<void> {
if (!this.proc?.stdin) {
throw new Error('piper-svc: stdin unavailable');
}
console.log('[TX]', line);
this.proc.stdin.write(line + '\n');
}
// ── pending-init promises (separate from synth to avoid clearing respMap on init failure) ──
private initResolve: (() => void) | null = null;
private initReject: ((e: Error) => void) | null = null;
private resolveInit(): void {
if (!this.pendingInit) return;
this.initResolve?.();
this.pendingInit = null;
this.initResolve = null;
this.initReject = null;
}
private rejectResponse(msgId: string, err: Error): void {
const entry = this.respMap.get(msgId);
this.respMap.delete(msgId);
if (entry) {
entry.reject(err);
}
}
private resolveInitError(err: Error): void {
if (!this.pendingInit) return;
this.initReject?.(err);
this.pendingInit = null;
this.initResolve = null;
this.initReject = null;
}
// ── pending synth responses (separate from init so init failure doesn't clear them) ──
private respMap = new Map<string, {
resolve: (wavPath: string) => void;
reject: (e: Error) => void;
}>();
private resolveResponse(msgId: string, wavPath: string): void {
const entry = this.respMap.get(msgId);
this.respMap.delete(msgId);
if (entry?.resolve) entry.resolve(wavPath);
}
// ── public spawn / initWav / synthWav ──
private async _spawn(): Promise<void> {
if (this.proc) return;
const workerPath = join(SCRIPT_DIR, 'piper-worker.py');
const venv = process.env.VIRTUAL_ENV || join(SCRIPT_DIR, '..', '.venv', 'bin', 'python3');
this.proc = spawn(venv, [workerPath], { stdio: ['pipe', 'pipe', 'pipe'] });
this.setupStdout();
if (this.proc.stderr) {
this.proc.stderr.on('data', (chunk: Buffer) => {
console.error(
'[piper-worker]',
chunk.toString().trim()
);
});
}
this.proc.on('exit', () => {
console.log('[piper-svc] Process exited, rejecting all pending synths');
// reject all pending (new format: {resolve, reject})
for (const [, entry] of this.respMap) entry.reject(new Error('piper process exited'));
this.respMap.clear();
if (this.pendingInit && this.initReject) {
this.initReject(new Error('piper process exited'));
this.pendingInit = null;
}
});
// ── cleanup old WAV files every 5 min ──
this.cleanupTimer = setInterval(() => {
try {
const now = Date.now();
for (const entry of readdirSync(TTS_DIR)) {
const fp = join(TTS_DIR, entry);
try {
const s = statSync(fp);
if (s.isFile() && now - s.mtimeMs > 300_000) unlinkSync(fp);
} catch { /* ignore */ }
}
} catch { /* ignore */ }
}, 5 * 60 * 1000);
}
async initWav(): Promise<void> {
if (!this.proc) await this._spawn();
if (this.pendingInit) return this.pendingInit;
this.pendingInit = new Promise<void>((resolve, reject) => {
this.initResolve = resolve;
this.initReject = reject;
});
const modelPath = getPiperModel() || join('/tmp', 'quibot-piper-models', 'ca_ES-upc_ona-medium.onnx');
const cfgPath = modelPath.replace(/\.onnx$/, '.onnx.json');
await this.writeStdin(
JSON.stringify({ type: 'init', model: modelPath, config: cfgPath }),
);
return this.pendingInit;
}
/** Synthesize with local Piper subprocess (primary) and HTTP Piper fallback */
async synthesize(params: { text: string }): Promise<Buffer> {
try {
await this.initWav();
return await this.synthWav(params.text);
} catch (localErr) {
const url = getPiperUrl();
if (url) {
console.log(`[tts] Local Piper failed: ${localErr instanceof Error ? localErr.message : localErr}. Falling back to remote.`);
return await httpPiperService.synthesize({ ...params, lang: 'ca' });
}
throw localErr;
}
}
async synthWav(text: string): Promise<Buffer> {
await this.initWav();
if (!this.proc) await this._spawn(); // auto-spawn; init runs concurrently
const msgId = randomUUID() + '-' + Date.now();
const outPath = join(TTS_DIR, `${msgId}.wav`);
return new Promise((resolve, reject) => {
let cleared = false;
const timer = setTimeout(() => {
if (cleared) return;
cleared = true;
this.respMap.delete(msgId);
reject(new Error('piper-svc: synthesis timed out after 120s'));
}, 120_000);
this.respMap.set(msgId, {
resolve: (wavPath: string) => {
if (cleared) return;
cleared = true;
clearTimeout(timer);
console.log(`[piper-svc] Synthesized ${wavPath} (${Date.now()})`);
try {
const buf = readFileSync(wavPath);
rmSync(wavPath, { force: true });
resolve(buf);
} catch (err: unknown) {
reject(err instanceof Error ? err : new Error('read WAV failed'));
}
},
reject: (e: Error) => {
if (cleared) return;
cleared = true;
clearTimeout(timer);
this.respMap.delete(msgId);
reject(e);
},
});
console.log(`[piper-svc] synthesize ${text.substring(0, 40)}... (msgId=${msgId})`);
this.writeStdin(
JSON.stringify({ type: 'synthesize', text, msgId, outPath }),
).catch((e) => {
if (cleared) return;
cleared = true;
this.respMap.delete(msgId);
reject(e instanceof Error ? e : new Error(String(e)));
});
});
}
// ── shutdown ──
async shutdown(): Promise<void> {
if (this.cleanupTimer) {
clearInterval(this.cleanupTimer);
this.cleanupTimer = null;
}
if (!this.proc) return;
const p = this.proc; this.proc = null;
await new Promise<void>((res) => {
p.on('exit', res);
setTimeout(res, 3000);
if (!p.killed) { try { p.stdin?.end(); } catch {} p.kill('SIGTERM'); }
});
}
}
// ─── singleton ────────────────────────────────────────────────
export const piperService = new PiperLocalService();

218
backend/src/services/whisper.service.ts
View File

@@ -1,218 +0,0 @@
import { spawn, ChildProcess } from 'child_process';
import { join } from 'path';
import { fileURLToPath } from 'url';
import { randomUUID } from 'crypto';
const __filename = fileURLToPath(import.meta.url);
const __dirname = join(__filename, '..');
const SCRIPT_DIR = join(__dirname, '..');
const PYTHON = join(SCRIPT_DIR, '..', '.venv', 'bin', 'python3');
const whisperModel = process.env.WHISPER_MODEL ?? 'small';
const whisperLanguage = process.env.WHISPER_LANGUAGE ?? 'ca';
interface TranscriptResult {
msgId: string;
text?: string;
error?: string;
}
interface InitResult {
type: 'init_ok' | 'init_error';
}
class WhisperService {
private proc: ChildProcess | null = null;
private onInitResolve: (() => void) | null = null;
private onInitReject: ((err: Error) => void) | null = null;
spawn(): void {
if (this.proc) return;
const scriptPath = join(SCRIPT_DIR, 'whisper-worker.py');
this.proc = spawn(PYTHON, [scriptPath], {
stdio: ['pipe', 'pipe', 'pipe'],
env: { ...process.env },
});
if (!this.proc.stdout || !this.proc.stderr || !this.proc.stdin) {
console.error('[whisper-svc] Missing stdin/stdout/stderr');
this.proc = null;
return;
}
const proc = this.proc;
if (!proc?.stdout) return;
let buf = '';
proc.stdout.on('data', (chunk: Buffer) => {
buf += chunk.toString();
while (true) {
const nl = buf.indexOf('\n');
if (nl === -1) break;
const line = buf.slice(0, nl).trim();
buf = buf.slice(nl + 1);
if (!line) continue;
try {
const msg = JSON.parse(line);
if (msg.type === 'ready') {
console.log('[whisper-svc] Worker ready, sending init...');
proc.stdin!.write(
JSON.stringify({ type: 'init', model: whisperModel, language: whisperLanguage }) + '\n',
);
} else if (msg.type === 'init_ok') {
console.log(`[whisper-svc] Model loaded (model=${whisperModel}, lang=${whisperLanguage})`);
if (this.onInitResolve) {
const r = this.onInitResolve;
this.onInitResolve = null;
this.onInitReject = null;
r();
}
} else if (msg.type === 'init_error') {
const err = new Error(`whisper-svc init failed: ${msg.error || 'unknown'}`);
if (this.onInitReject) {
const r = this.onInitReject;
this.onInitResolve = null;
this.onInitReject = null;
r(err);
}
} else if (msg.type === 'transcript' || msg.type === 'error') {
this.resolveTranscript(msg.msgId, msg);
}
} catch { /* skip */ }
}
});
const stderr = proc.stderr;
if (stderr) {
stderr.on('data', (chunk: Buffer) => {
const text = chunk.toString().trim();
if (text) console.log(`[whisper-svc] stderr: ${text}`);
});
}
proc.on('exit', (code, signal) => {
console.log(`[whisper-svc] Exited code=${code} signal=${signal}`);
this.proc = null;
});
proc.on('error', (err) => {
console.error(`[whisper-svc] Error: ${err.message}`);
this.proc = null;
});
}
private pending: Map<string, (result: TranscriptResult) => void> = new Map();
private resolveTranscript(msgId: string, msg: { type?: string; text?: string; error?: string }) {
const cb = this.pending.get(msgId);
this.pending.delete(msgId);
if (cb) {
if (msg.type === 'error') {
cb({
msgId,
text: msg.text,
error: msg.error ?? msg.text ?? 'unknown error',
});
} else {
cb({ msgId, text: msg.text });
}
}
}
private waitForInit(): Promise<void> {
if (this.onInitResolve) return Promise.resolve(); // already initializing
return new Promise<void>((resolve, reject) => {
let cleared = false;
const timer = setTimeout(() => {
if (cleared) return;
cleared = true;
this.onInitReject = null;
reject(new Error('whisper-svc init timed out'));
}, 90_000);
this.onInitResolve = () => {
if (cleared) return;
cleared = true;
clearTimeout(timer);
resolve();
};
this.onInitReject = (err: Error) => {
if (cleared) return;
cleared = true;
clearTimeout(timer);
reject(err);
};
});
}
async transcribe(audioPath: string): Promise<string> {
if (!this.proc) {
this.spawn();
}
// await this.waitForInit();
const msgId = randomUUID() + '-' + Date.now();
return new Promise((resolve, reject) => {
let cleared = false;
let timer: ReturnType<typeof setTimeout> | null = null;
const resolvePromise = (result: TranscriptResult) => {
if (cleared) return;
cleared = true;
if (timer) clearTimeout(timer);
if (result.error) {
reject(new Error(`whisper-svc: ${result.error}`));
} else if (result.text) {
resolve(result.text.trim());
} else {
reject(new Error('whisper-svc: empty response'));
}
};
this.pending.set(msgId, resolvePromise);
timer = setTimeout(() => {
if (cleared) return;
cleared = true;
this.pending.delete(msgId);
reject(new Error('whisper-svc: transcription timed out'));
}, 120_000);
const proc = this.proc;
if (proc && proc.stdin) {
proc.stdin.write(
JSON.stringify({ type: 'transcribe', path: audioPath, msgId }) + '\n',
);
} else {
cleared = true;
if (timer) clearTimeout(timer);
this.pending.delete(msgId);
reject(new Error('whisper subprocess not running'));
}
});
}
async shutdown(): Promise<void> {
const proc = this.proc;
if (proc) {
try {
proc.stdin!.end();
await new Promise<void>((resolve) => {
proc.on('exit', () => resolve());
setTimeout(() => {
if (!proc.killed) proc.kill('SIGTERM');
resolve();
}, 3000);
});
} catch { /* ignore */ }
this.proc = null;
}
}
}
export const whisperService = new WhisperService();

55
backend/src/whisper-worker.py
View File

@@ -1,55 +0,0 @@
#!/usr/bin/env python3
"""Persistent Whisper transcription worker – single subprocess, model loaded once."""
import sys
import json
def main():
from faster_whisper import WhisperModel
model_path = "base"
language = "ca"
model = None
# Signal node that the process is alive and listening
print(json.dumps({"type": "ready"}), flush=True)
for line in sys.stdin:
line = line.strip()
if not line:
continue
try:
msg = json.loads(line)
except json.JSONDecodeError:
continue
if msg.get("type") == "init":
model_path = msg.get("model", "base")
language = msg.get("language", "ca") or "ca"
print(f"[whisper-worker] Loading model='{model_path}' language='{language}'", file=sys.stderr, flush=True)
model = WhisperModel(model_path, device="cpu", compute_type="int8")
print(json.dumps({"type": "init_ok"}), flush=True)
continue
if msg.get("type") == "transcribe":
audio_path = msg.get("path")
msg_id = msg.get("msgId", "")
if not audio_path:
print(json.dumps({"type": "error", "text": "no path provided", "msgId": msg_id}), flush=True)
continue
try:
segments, info = model.transcribe(audio_path, language=language or None)
transcript = ""
for seg in segments:
transcript += seg.text + " "
result_text = transcript.strip()
print(json.dumps({"type": "transcript", "text": result_text, "msgId": msg_id}), flush=True)
except Exception as exc:
print(json.dumps({"type": "error", "text": str(exc), "msgId": msg_id}), flush=True)
if __name__ == "__main__":
main()

1
mcp/.gitignore vendored
View File

@@ -1 +0,0 @@
.venv/

46
mcp/mcp_server.py
View File

@@ -1,46 +0,0 @@
from fastmcp import FastMCP
import requests
import threading
mcp = FastMCP("Test MCP server")
def fire_and_forget(url: str):
try:
requests.get(url, timeout=2)
except Exception:
pass # ignore all errors so it never affects the tool
@mcp.tool()
def add(a: int, b: int) -> int:
"""Add two numbers together"""
return a + b
@mcp.tool()
def greet(name: str) -> str:
"""Greet someome by its name"""
return f"Hello {name}! Welcome!"
@mcp.tool()
def multiply(a: int, b: int) -> int:
"""Multiply two numbers"""
return a * b
@mcp.tool()
def get_time() -> str:
"""Get the current time"""
from datetime import datetime
return datetime.now().strftime("%I:%M %p")
@mcp.tool()
def moure_brac() -> str:
"""Mou els braços"""
url = "http://quibot.local:8000/greet"
# start background request (non-blocking)
threading.Thread(target=fire_and_forget, args=(url,), daemon=True).start()
return "Braços moguts"
if __name__ == "__main__":
mcp.run(transport="sse", port=5001)

10
package-lock.json generated
View File

@@ -5,7 +5,6 @@
"packages": { "packages": {
"": { "": {
"dependencies": { "dependencies": {
"formdata-node": "^6.0.3",
"vue-i18n": "^11.4.5" "vue-i18n": "^11.4.5"
} }
}, },
@@ -269,15 +268,6 @@
"license": "MIT", "license": "MIT",
"peer": true "peer": true
}, },
"node_modules/formdata-node": {
"version": "6.0.3",
"resolved": "https://registry.npmjs.org/formdata-node/-/formdata-node-6.0.3.tgz",
"integrity": "sha512-8e1++BCiTzUno9v5IZ2J6bv4RU+3UKDmqWUQD0MIMVCd9AdhWkO1gw57oo1mNEX1dMq2EGI+FbWz4B92pscSQg==",
"license": "MIT",
"engines": {
"node": ">= 18"
}
},
"node_modules/magic-string": { "node_modules/magic-string": {
"version": "0.30.21", "version": "0.30.21",
"resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz", "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz",

1
package.json
View File

@@ -1,6 +1,5 @@
{ {
"dependencies": { "dependencies": {
"formdata-node": "^6.0.3",
"vue-i18n": "^11.4.5" "vue-i18n": "^11.4.5"
} }
} }

19
quibot-web/android/.gitignore vendored Normal file
View File

@@ -0,0 +1,19 @@
# OSX
#
.DS_Store
# Android/IntelliJ
#
build/
.idea
.gradle
local.properties
*.iml
*.hprof
.cxx/
# generated inline modules
app/src/main/java/inline/
# Bundle artifacts
*.jsbundle

182
quibot-web/android/app/build.gradle Normal file
View File

@@ -0,0 +1,182 @@
apply plugin: "com.android.application"
apply plugin: "org.jetbrains.kotlin.android"
apply plugin: "com.facebook.react"
def projectRoot = rootDir.getAbsoluteFile().getParentFile().getAbsolutePath()
/**
* This is the configuration block to customize your React Native Android app.
* By default you don't need to apply any configuration, just uncomment the lines you need.
*/
react {
entryFile = file(["node", "-e", "require('expo/scripts/resolveAppEntry')", projectRoot, "android", "absolute"].execute(null, rootDir).text.trim())
reactNativeDir = new File(["node", "--print", "require.resolve('react-native/package.json')"].execute(null, rootDir).text.trim()).getParentFile().getAbsoluteFile()
hermesCommand = new File(["node", "--print", "require.resolve('hermes-compiler/package.json', { paths: [require.resolve('react-native/package.json')] })"].execute(null, rootDir).text.trim()).getParentFile().getAbsolutePath() + "/hermesc/%OS-BIN%/hermesc"
codegenDir = new File(["node", "--print", "require.resolve('@react-native/codegen/package.json', { paths: [require.resolve('react-native/package.json')] })"].execute(null, rootDir).text.trim()).getParentFile().getAbsoluteFile()
enableBundleCompression = (findProperty('android.enableBundleCompression') ?: false).toBoolean()
// Use Expo CLI to bundle the app, this ensures the Metro config
// works correctly with Expo projects.
cliFile = new File(["node", "--print", "require.resolve('@expo/cli', { paths: [require.resolve('expo/package.json')] })"].execute(null, rootDir).text.trim())
bundleCommand = "export:embed"
/* Folders */
// The root of your project, i.e. where "package.json" lives. Default is '../..'
// root = file("../../")
// The folder where the react-native NPM package is. Default is ../../node_modules/react-native
// reactNativeDir = file("../../node_modules/react-native")
// The folder where the react-native Codegen package is. Default is ../../node_modules/@react-native/codegen
// codegenDir = file("../../node_modules/@react-native/codegen")
/* Variants */
// The list of variants to that are debuggable. For those we're going to
// skip the bundling of the JS bundle and the assets. By default is just 'debug'.
// If you add flavors like lite, prod, etc. you'll have to list your debuggableVariants.
// debuggableVariants = ["liteDebug", "prodDebug"]
/* Bundling */
// A list containing the node command and its flags. Default is just 'node'.
// nodeExecutableAndArgs = ["node"]
//
// The path to the CLI configuration file. Default is empty.
// bundleConfig = file(../rn-cli.config.js)
//
// The name of the generated asset file containing your JS bundle
// bundleAssetName = "MyApplication.android.bundle"
//
// The entry file for bundle generation. Default is 'index.android.js' or 'index.js'
// entryFile = file("../js/MyApplication.android.js")
//
// A list of extra flags to pass to the 'bundle' commands.
// See https://github.com/react-native-community/cli/blob/main/docs/commands.md#bundle
// extraPackagerArgs = []
/* Hermes Commands */
// The hermes compiler command to run. By default it is 'hermesc'
// hermesCommand = "$rootDir/my-custom-hermesc/bin/hermesc"
//
// The list of flags to pass to the Hermes compiler. By default is "-O", "-output-source-map"
// hermesFlags = ["-O", "-output-source-map"]
/* Autolinking */
autolinkLibrariesWithApp()
}
/**
* Set this to true in release builds to optimize the app using [R8](https://developer.android.com/topic/performance/app-optimization/enable-app-optimization).
*/
def enableMinifyInReleaseBuilds = (findProperty('android.enableMinifyInReleaseBuilds') ?: false).toBoolean()
/**
* The preferred build flavor of JavaScriptCore (JSC)
*
* For example, to use the international variant, you can use:
* `def jscFlavor = 'org.webkit:android-jsc-intl:+'`
*
* The international variant includes ICU i18n library and necessary data
* allowing to use e.g. `Date.toLocaleString` and `String.localeCompare` that
* give correct results when using with locales other than en-US. Note that
* this variant is about 6MiB larger per architecture than default.
*/
def jscFlavor = 'io.github.react-native-community:jsc-android:2026004.+'
android {
ndkVersion rootProject.ext.ndkVersion
buildToolsVersion rootProject.ext.buildToolsVersion
compileSdk rootProject.ext.compileSdkVersion
namespace 'com.arandano69.quibotweb'
defaultConfig {
applicationId 'com.arandano69.quibotweb'
minSdkVersion rootProject.ext.minSdkVersion
targetSdkVersion rootProject.ext.targetSdkVersion
versionCode 1
versionName "1.0.0"
buildConfigField "String", "REACT_NATIVE_RELEASE_LEVEL", "\"${findProperty('reactNativeReleaseLevel') ?: 'stable'}\""
}
signingConfigs {
debug {
storeFile file('debug.keystore')
storePassword 'android'
keyAlias 'androiddebugkey'
keyPassword 'android'
}
}
buildTypes {
debug {
signingConfig signingConfigs.debug
}
release {
// Caution! In production, you need to generate your own keystore file.
// see https://reactnative.dev/docs/signed-apk-android.
signingConfig signingConfigs.debug
def enableShrinkResources = findProperty('android.enableShrinkResourcesInReleaseBuilds') ?: 'false'
shrinkResources enableShrinkResources.toBoolean()
minifyEnabled enableMinifyInReleaseBuilds
proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
def enablePngCrunchInRelease = findProperty('android.enablePngCrunchInReleaseBuilds') ?: 'true'
crunchPngs enablePngCrunchInRelease.toBoolean()
}
}
packagingOptions {
jniLibs {
def enableLegacyPackaging = findProperty('expo.useLegacyPackaging') ?: 'false'
useLegacyPackaging enableLegacyPackaging.toBoolean()
}
}
androidResources {
ignoreAssetsPattern '!.svn:!.git:!.ds_store:!*.scc:!CVS:!thumbs.db:!picasa.ini:!*~'
}
}
// Apply static values from `gradle.properties` to the `android.packagingOptions`
// Accepts values in comma delimited lists, example:
// android.packagingOptions.pickFirsts=/LICENSE,**/picasa.ini
["pickFirsts", "excludes", "merges", "doNotStrip"].each { prop ->
// Split option: 'foo,bar' -> ['foo', 'bar']
def options = (findProperty("android.packagingOptions.$prop") ?: "").split(",");
// Trim all elements in place.
for (i in 0..<options.size()) options[i] = options[i].trim();
// `[] - ""` is essentially `[""].filter(Boolean)` removing all empty strings.
options -= ""
if (options.length > 0) {
println "android.packagingOptions.$prop += $options ($options.length)"
// Ex: android.packagingOptions.pickFirsts += '**/SCCS/**'
options.each {
android.packagingOptions[prop] += it
}
}
}
dependencies {
// The version of react-native is set by the React Native Gradle Plugin
implementation("com.facebook.react:react-android")
def isGifEnabled = (findProperty('expo.gif.enabled') ?: "") == "true";
def isWebpEnabled = (findProperty('expo.webp.enabled') ?: "") == "true";
def isWebpAnimatedEnabled = (findProperty('expo.webp.animated') ?: "") == "true";
if (isGifEnabled) {
// For animated gif support
implementation("com.facebook.fresco:animated-gif:${expoLibs.versions.fresco.get()}")
}
if (isWebpEnabled) {
// For webp support
implementation("com.facebook.fresco:webpsupport:${expoLibs.versions.fresco.get()}")
if (isWebpAnimatedEnabled) {
// Animated webp support
implementation("com.facebook.fresco:animated-webp:${expoLibs.versions.fresco.get()}")
}
}
if (hermesEnabled.toBoolean()) {
implementation("com.facebook.react:hermes-android")
} else {
implementation jscFlavor
}
}

BIN
quibot-web/android/app/debug.keystore Normal file
View File

Binary file not shown.

14
quibot-web/android/app/proguard-rules.pro vendored Normal file
View File

@@ -0,0 +1,14 @@
# Add project specific ProGuard rules here.
# By default, the flags in this file are appended to flags specified
# in /usr/local/Cellar/android-sdk/24.3.3/tools/proguard/proguard-android.txt
# You can edit the include path and order by changing the proguardFiles
# directive in build.gradle.
#
# For more details, see
# http://developer.android.com/guide/developing/tools/proguard.html
# react-native-reanimated
-keep class com.swmansion.reanimated.** { *; }
-keep class com.facebook.react.turbomodule.** { *; }
# Add any project specific keep options here:

7
quibot-web/android/app/src/debug/AndroidManifest.xml Normal file
View File

@@ -0,0 +1,7 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW"/>
<application android:usesCleartextTraffic="true" tools:targetApi="28" tools:ignore="GoogleAppIndexingWarning" tools:replace="android:usesCleartextTraffic" />
</manifest>

7
quibot-web/android/app/src/debugOptimized/AndroidManifest.xml Normal file
View File

@@ -0,0 +1,7 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW"/>
<application android:usesCleartextTraffic="true" tools:targetApi="28" tools:ignore="GoogleAppIndexingWarning" tools:replace="android:usesCleartextTraffic" />
</manifest>

26
quibot-web/android/app/src/main/AndroidManifest.xml Normal file
View File

@@ -0,0 +1,26 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools">
<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" android:maxSdkVersion="32" tools:replace="android:maxSdkVersion"/>
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW"/>
<uses-permission android:name="android.permission.VIBRATE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" android:maxSdkVersion="32" tools:replace="android:maxSdkVersion"/>
<queries>
<intent>
<action android:name="android.intent.action.VIEW"/>
<category android:name="android.intent.category.BROWSABLE"/>
<data android:scheme="https"/>
</intent>
</queries>
<application android:name=".MainApplication" android:label="@string/app_name" android:icon="@mipmap/ic_launcher" android:roundIcon="@mipmap/ic_launcher_round" android:allowBackup="true" android:theme="@style/AppTheme" android:supportsRtl="true" android:enableOnBackInvokedCallback="false">
<meta-data android:name="expo.modules.updates.ENABLED" android:value="false"/>
<meta-data android:name="expo.modules.updates.ENABLE_BSDIFF_PATCH_SUPPORT" android:value="true"/>
<meta-data android:name="expo.modules.updates.EXPO_UPDATES_CHECK_ON_LAUNCH" android:value="ALWAYS"/>
<meta-data android:name="expo.modules.updates.EXPO_UPDATES_LAUNCH_WAIT_MS" android:value="0"/>
<activity android:name=".MainActivity" android:configChanges="keyboard|keyboardHidden|orientation|screenSize|screenLayout|uiMode|smallestScreenSize" android:launchMode="singleTask" android:windowSoftInputMode="adjustResize" android:theme="@style/Theme.App.SplashScreen" android:exported="true">
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
</intent-filter>
</activity>
</application>
</manifest>

61
quibot-web/android/app/src/main/java/com/arandano69/quibotweb/MainActivity.kt Normal file
View File

@@ -0,0 +1,61 @@
package com.arandano69.quibotweb
import android.os.Build
import android.os.Bundle
import com.facebook.react.ReactActivity
import com.facebook.react.ReactActivityDelegate
import com.facebook.react.defaults.DefaultNewArchitectureEntryPoint.fabricEnabled
import com.facebook.react.defaults.DefaultReactActivityDelegate
import expo.modules.ReactActivityDelegateWrapper
class MainActivity : ReactActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
// Set the theme to AppTheme BEFORE onCreate to support
// coloring the background, status bar, and navigation bar.
// This is required for expo-splash-screen.
setTheme(R.style.AppTheme);
super.onCreate(null)
}
/**
* Returns the name of the main component registered from JavaScript. This is used to schedule
* rendering of the component.
*/
override fun getMainComponentName(): String = "main"
/**
* Returns the instance of the [ReactActivityDelegate]. We use [DefaultReactActivityDelegate]
* which allows you to enable New Architecture with a single boolean flags [fabricEnabled]
*/
override fun createReactActivityDelegate(): ReactActivityDelegate {
return ReactActivityDelegateWrapper(
this,
BuildConfig.IS_NEW_ARCHITECTURE_ENABLED,
object : DefaultReactActivityDelegate(
this,
mainComponentName,
fabricEnabled
){})
}
/**
* Align the back button behavior with Android S
* where moving root activities to background instead of finishing activities.
* @see <a href="https://developer.android.com/reference/android/app/Activity#onBackPressed()">onBackPressed</a>
*/
override fun invokeDefaultOnBackPressed() {
if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.R) {
if (!moveTaskToBack(false)) {
// For non-root activities, use the default implementation to finish them.
super.invokeDefaultOnBackPressed()
}
return
}
// Use the default back button implementation on Android S
// because it's doing more than [Activity.moveTaskToBack] in fact.
super.invokeDefaultOnBackPressed()
}
}

45
quibot-web/android/app/src/main/java/com/arandano69/quibotweb/MainApplication.kt Normal file
View File

@@ -0,0 +1,45 @@
package com.arandano69.quibotweb
import android.app.Application
import android.content.res.Configuration
import com.facebook.react.PackageList
import com.facebook.react.ReactApplication
import com.facebook.react.ReactNativeApplicationEntryPoint.loadReactNative
import com.facebook.react.ReactPackage
import com.facebook.react.ReactHost
import com.facebook.react.common.ReleaseLevel
import com.facebook.react.defaults.DefaultNewArchitectureEntryPoint
import expo.modules.ApplicationLifecycleDispatcher
import expo.modules.ExpoReactHostFactory
class MainApplication : Application(), ReactApplication {
override val reactHost: ReactHost by lazy {
ExpoReactHostFactory.getDefaultReactHost(
context = applicationContext,
packageList =
PackageList(this).packages.apply {
// Packages that cannot be autolinked yet can be added manually here, for example:
// add(MyReactNativePackage())
}
)
}
override fun onCreate() {
super.onCreate()
DefaultNewArchitectureEntryPoint.releaseLevel = try {
ReleaseLevel.valueOf(BuildConfig.REACT_NATIVE_RELEASE_LEVEL.uppercase())
} catch (e: IllegalArgumentException) {
ReleaseLevel.STABLE
}
loadReactNative(this)
ApplicationLifecycleDispatcher.onApplicationCreate(this)
}
override fun onConfigurationChanged(newConfig: Configuration) {
super.onConfigurationChanged(newConfig)
ApplicationLifecycleDispatcher.onConfigurationChanged(this, newConfig)
}
}

BIN
quibot-web/android/app/src/main/res/drawable-hdpi/splashscreen_logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

BIN
quibot-web/android/app/src/main/res/drawable-mdpi/splashscreen_logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

BIN
quibot-web/android/app/src/main/res/drawable-xhdpi/splashscreen_logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 28 KiB

BIN
quibot-web/android/app/src/main/res/drawable-xxhdpi/splashscreen_logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 46 KiB

BIN
quibot-web/android/app/src/main/res/drawable-xxxhdpi/splashscreen_logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 65 KiB

6
quibot-web/android/app/src/main/res/drawable/ic_launcher_background.xml Normal file
View File

@@ -0,0 +1,6 @@
<layer-list xmlns:android="http://schemas.android.com/apk/res/android">
<item android:drawable="@color/splashscreen_background"/>
<item>
<bitmap android:gravity="center" android:src="@drawable/splashscreen_logo"/>
</item>
</layer-list>

37
quibot-web/android/app/src/main/res/drawable/rn_edit_text_material.xml Normal file
View File

@@ -0,0 +1,37 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (C) 2014 The Android Open Source Project
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<inset xmlns:android="http://schemas.android.com/apk/res/android"
android:insetLeft="@dimen/abc_edit_text_inset_horizontal_material"
android:insetRight="@dimen/abc_edit_text_inset_horizontal_material"
android:insetTop="@dimen/abc_edit_text_inset_top_material"
android:insetBottom="@dimen/abc_edit_text_inset_bottom_material"
>
<selector>
<!--
This file is a copy of abc_edit_text_material (https://bit.ly/3k8fX7I).
The item below with state_pressed="false" and state_focused="false" causes a NullPointerException.
NullPointerException:tempt to invoke virtual method 'android.graphics.drawable.Drawable android.graphics.drawable.Drawable$ConstantState.newDrawable(android.content.res.Resources)'
<item android:state_pressed="false" android:state_focused="false" android:drawable="@drawable/abc_textfield_default_mtrl_alpha"/>
For more info, see https://bit.ly/3CdLStv (react-native/pull/29452) and https://bit.ly/3nxOMoR.
-->
<item android:state_enabled="false" android:drawable="@drawable/abc_textfield_default_mtrl_alpha"/>
<item android:drawable="@drawable/abc_textfield_activated_mtrl_alpha"/>
</selector>
</inset>

BIN
quibot-web/android/app/src/main/res/mipmap-hdpi/ic_launcher.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.0 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.9 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-mdpi/ic_launcher.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.0 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.8 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-xhdpi/ic_launcher.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.5 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.9 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.3 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.0 KiB

BIN
quibot-web/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

4
quibot-web/android/app/src/main/res/values/colors.xml Normal file
View File

@@ -0,0 +1,4 @@
<resources>
<color name="splashscreen_background">#FFFFFF</color>
<color name="colorPrimary">#023c69</color>
</resources>

3
quibot-web/android/app/src/main/res/values/strings.xml Normal file
View File

@@ -0,0 +1,3 @@
<resources>
<string name="app_name">quibot-web</string>
</resources>

11
quibot-web/android/app/src/main/res/values/styles.xml Normal file
View File

@@ -0,0 +1,11 @@
<resources xmlns:tools="http://schemas.android.com/tools">
<style name="AppTheme" parent="Theme.AppCompat.DayNight.NoActionBar">
<item name="android:editTextBackground">@drawable/rn_edit_text_material</item>
<item name="colorPrimary">@color/colorPrimary</item>
<item name="android:statusBarColor">@android:color/transparent</item>
<item name="android:navigationBarColor">@android:color/transparent</item>
</style>
<style name="Theme.App.SplashScreen" parent="AppTheme">
<item name="android:windowBackground">@drawable/splashscreen_logo</item>
</style>
</resources>

24
quibot-web/android/build.gradle Normal file
View File

@@ -0,0 +1,24 @@
// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
repositories {
google()
mavenCentral()
}
dependencies {
classpath('com.android.tools.build:gradle')
classpath('com.facebook.react:react-native-gradle-plugin')
classpath('org.jetbrains.kotlin:kotlin-gradle-plugin')
}
}
allprojects {
repositories {
google()
mavenCentral()
maven { url 'https://www.jitpack.io' }
}
}
apply plugin: "expo-root-project"
apply plugin: "com.facebook.react.rootproject"

63
quibot-web/android/gradle.properties Normal file
View File

@@ -0,0 +1,63 @@
# Project-wide Gradle settings.
# IDE (e.g. Android Studio) users:
# Gradle settings configured through the IDE *will override*
# any settings specified in this file.
# For more details on how to configure your build environment visit
# http://www.gradle.org/docs/current/userguide/build_environment.html
# Specifies the JVM arguments used for the daemon process.
# The setting is particularly useful for tweaking memory settings.
# Default value: -Xmx512m -XX:MaxMetaspaceSize=256m
org.gradle.jvmargs=-Xmx2048m -XX:MaxMetaspaceSize=512m
# When configured, Gradle will run in incubating parallel mode.
# This option should only be used with decoupled projects. More details, visit
# http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:decoupled_projects
org.gradle.parallel=true
# AndroidX package structure to make it clearer which packages are bundled with the
# Android operating system, and which are packaged with your app's APK
# https://developer.android.com/topic/libraries/support-library/androidx-rn
android.useAndroidX=true
# Enable AAPT2 PNG crunching
android.enablePngCrunchInReleaseBuilds=true
# Use this property to specify which architecture you want to build.
# You can also override it from the CLI using
# ./gradlew <task> -PreactNativeArchitectures=x86_64
reactNativeArchitectures=armeabi-v7a,arm64-v8a,x86,x86_64
# Use this property to enable support to the new architecture.
# This will allow you to use TurboModules and the Fabric render in
# your application. You should enable this flag either if you want
# to write custom TurboModules/Fabric components OR use libraries that
# are providing them.
newArchEnabled=true
# Use this property to enable or disable the Hermes JS engine.
# If set to false, you will be using JSC instead.
hermesEnabled=true
# Use this property to enable edge-to-edge display support.
# This allows your app to draw behind system bars for an immersive UI.
# Note: Only works with ReactActivity and should not be used with custom Activity.
edgeToEdgeEnabled=true
# Enable GIF support in React Native images (~200 B increase)
expo.gif.enabled=true
# Enable webp support in React Native images (~85 KB increase)
expo.webp.enabled=true
# Enable animated webp support (~3.4 MB increase)
# Disabled by default because iOS doesn't support animated webp
expo.webp.animated=false
# Enable network inspector
EX_DEV_CLIENT_NETWORK_INSPECTOR=true
# Use legacy packaging to compress native libraries in the resulting APK.
expo.useLegacyPackaging=false
expo.inlineModules.watchedDirectories=[]

BIN
quibot-web/android/gradle/wrapper/gradle-wrapper.jar vendored Normal file
View File

Binary file not shown.

7
quibot-web/android/gradle/wrapper/gradle-wrapper.properties vendored Normal file
View File

@@ -0,0 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-9.3.1-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

248
quibot-web/android/gradlew vendored Executable file
View File

@@ -0,0 +1,248 @@
#!/bin/sh
#
# Copyright © 2015 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
#
##############################################################################
#
# Gradle start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh Gradle
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
# This is normally unused
# shellcheck disable=SC2034
APP_BASE_NAME=${0##*/}
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD=maximum
warn () {
echo "$*"
} >&2
die () {
echo
echo "$*"
echo
exit 1
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD=java
if ! command -v java >/dev/null 2>&1
then
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
fi
# Increase the maximum file descriptors if we can.
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Collect all arguments for the java command:
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
# and any embedded shellness will be escaped.
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
# treated as '${Hostname}' itself on the command line.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"

98
quibot-web/android/gradlew.bat vendored Normal file
View File

@@ -0,0 +1,98 @@
@REM Copyright (c) Meta Platforms, Inc. and affiliates.
@REM
@REM This source code is licensed under the MIT license found in the
@REM LICENSE file in the root directory of this source tree.
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@rem SPDX-License-Identifier: Apache-2.0
@rem
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if %ERRORLEVEL% equ 0 goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:execute
@rem Setup the command line
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %*
:end
@rem End local scope for the variables with windows NT shell
if %ERRORLEVEL% equ 0 goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega

39
quibot-web/android/settings.gradle Normal file
View File

@@ -0,0 +1,39 @@
pluginManagement {
def reactNativeGradlePlugin = new File(
providers.exec {
workingDir(rootDir)
commandLine("node", "--print", "require.resolve('@react-native/gradle-plugin/package.json', { paths: [require.resolve('react-native/package.json')] })")
}.standardOutput.asText.get().trim()
).getParentFile().absolutePath
includeBuild(reactNativeGradlePlugin)
def expoPluginsPath = new File(
providers.exec {
workingDir(rootDir)
commandLine("node", "--print", "require.resolve('expo-modules-autolinking/package.json', { paths: [require.resolve('expo/package.json')] })")
}.standardOutput.asText.get().trim(),
"../android/expo-gradle-plugin"
).absolutePath
includeBuild(expoPluginsPath)
}
plugins {
id("com.facebook.react.settings")
id("expo-autolinking-settings")
}
extensions.configure(com.facebook.react.ReactSettingsExtension) { ex ->
if (System.getenv('EXPO_USE_COMMUNITY_AUTOLINKING') == '1') {
ex.autolinkLibrariesFromCommand()
} else {
ex.autolinkLibrariesFromCommand(expoAutolinking.rnConfigCommand)
}
}
expoAutolinking.useExpoModules()
rootProject.name = 'quibot-web'
expoAutolinking.useExpoVersionCatalog()
include ':app'
includeBuild(expoAutolinking.reactNativeGradlePlugin)

5
quibot-web/app.json Normal file
View File

@@ -0,0 +1,5 @@
{
"android": {
"package": "com.arandano69.quibotweb"
}
}

3531
quibot-web/package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

9
quibot-web/package.json
View File

@@ -10,9 +10,14 @@
"postinstall": "nuxt prepare" "postinstall": "nuxt prepare"
}, },
"dependencies": { "dependencies": {
"expo-speech-recognition": "^56.0.1",
"nuxt": "^4.4.2", "nuxt": "^4.4.2",
"react-native-svg": "^15.15.5", "react-native-svg": "^15.15.5",
"vue": "^3.5.32", "vue": "^3.5.32",
"vue-router": "^5.0.4" "vue-router": "^5.0.4",
} "expo": "~56.0.12",
"react": "19.2.3",
"react-native": "0.85.3"
},
"version": "1.0.0"
} }

81
rasp/server.py
View File

@@ -1,81 +0,0 @@
from flask import Flask, request, jsonify
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'Rasp'))
import time
import pigpio
import motion
from motion import (
motion_setup, motion_setup_steppers, motion_setup_sensors, motion_cleanup,
enable_wheels, enable_arms, enable_syringe,
arms_home, syringe_home,
distance_to_object,
ON, OFF, CW, CCW,
)
def _pi_connect():
pi = pigpio.pi()
if not pi.connected:
print("ERROR: pigpiod no està en marxa. Executa: sudo pigpiod -s 1")
sys.exit(1)
return pi
def _setup_motors():
"""Setup mínim per a tests de motors (sense sensors I2C)."""
pi = _pi_connect()
motion_setup_steppers(pi)
return pi
def _setup_sensors():
"""Setup mínim per a tests de sensors I2C (sense steppers)."""
pi = _pi_connect()
motion_setup_sensors(pi)
return pi
def _teardown_motors(pi):
motion_cleanup()
pi.stop()
def _teardown_sensors(pi):
pi.stop()
def test_arms_pair():
pi = _setup_motors()
enable_arms(ON); time.sleep(0.1)
motion.arm_R.move(+200); motion.arm_L.move(+200)
time.sleep(1.5)
motion.arm_R.move(-200); motion.arm_L.move(-200)
time.sleep(0.5)
enable_arms(OFF)
app = Flask(__name__)
@app.route("/")
def home():
return jsonify({
"message": "Flask API is running"
})
@app.route("/greet", methods=["GET"])
def greet():
test_arms_pair()
return jsonify({
"message": "Hello, world!"
})
# Simple error handlers
@app.errorhandler(404)
def not_found(e):
return jsonify({"error": "Route not found"}), 404
@app.errorhandler(500)
def server_error(e):
return jsonify({"error": "Internal server error"}), 500
if __name__ == "__main__":
app.run(debug=True, host="0.0.0.0", port=8000)

2
raspi/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
__pycache__/
venv/

160
raspi/blocks.py Normal file
View File

@@ -0,0 +1,160 @@
"""
blocks.py — Lectura del sensor de color TCS34725 i servo d'expulsió de blocs.
Equivalent a blocks.cpp del codi Arduino/ESP32.
Requereix /boot/config.txt:
dtoverlay=i2c-gpio,bus=4,i2c_gpio_sda=22,i2c_gpio_scl=27
"""
import time
import pigpio
import adafruit_extended_bus
import adafruit_tcs34725
from pins import SERVO_PWM
# ==================
# IDs de color
# ==================
BK = 0 # Negre (no reconegut)
RD = 1 # Vermell → avançar
GN = 2 # Verd → girar dreta
BU = 3 # Blau → girar esquerra
YE = 4 # Groc → xuclar líquid
OG = 5 # Taronja → buidar líquid
VT = 6 # Violeta → sorpresa
NUM_COLORS = 7
# Taula de colors de referència (valors RGB 0–255, calibrats amb el sensor)
_COLORS = [
{"name": "BK", "r": 80, "g": 80, "b": 80},
{"name": "RD", "r": 202, "g": 32, "b": 34},
{"name": "GN", "r": 107, "g": 90, "b": 57},
{"name": "BU", "r": 104, "g": 83, "b": 66},
{"name": "YE", "r": 150, "g": 69, "b": 33},
{"name": "OG", "r": 185, "g": 44, "b": 32},
{"name": "VT", "r": 129, "g": 70, "b": 55},
]
# ==================
# Paràmetres del servo
# ==================
# Valors en µs de pulse width (pigpio set_servo_pulsewidth).
# Conversió des de l'ESP32 (16 bits, 50Hz): valor/65535 * 20000µs
# MIN = 3277/65535 * 20000 ≈ 1000µs
# MAX = 8000/65535 * 20000 ≈ 2440µs
# EJECT= 6450/65535 * 20000 ≈ 1968µs
MIN_SERVO_US = 1000 # µs → ~0°
MAX_SERVO_US = 2440 # µs → ~180°
OPEN_POSITION = 2440 # µs (equivalent a 8000 ESP32)
EJECT_POSITION = 1968 # µs (equivalent a 6450 ESP32)
_INCREMENT_US = 3 # µs per iteració de 1ms (equivalent a increment de 10 ESP32)
# ==================
# Instàncies globals
# ==================
_pi: pigpio.pi = None
_color_sensor = None
_current_servo_pos: int = OPEN_POSITION
# ==================
# Setup
# ==================
def blocks_setup(pi: pigpio.pi):
"""
Inicialitza el servo i el sensor de color TCS34725.
El servo arranca en OPEN_POSITION.
"""
global _pi, _color_sensor, _current_servo_pos
_pi = pi
# Servo
pi.set_mode(SERVO_PWM, pigpio.OUTPUT)
_current_servo_pos = OPEN_POSITION
pi.set_servo_pulsewidth(SERVO_PWM, _current_servo_pos)
# Sensor de color TCS34725 via bus I2C 4 (bit-bang GPIO22=SDA, GPIO27=SCL)
i2c = adafruit_extended_bus.ExtendedI2C(4)
_color_sensor = adafruit_tcs34725.TCS34725(i2c)
_color_sensor.integration_time = 50 # ms
_color_sensor.gain = 4 # 4x (equivalent a TCS34725::Gain::X04)
# ==================
# Servo
# ==================
def servo_move_to(target_us: int):
"""
Mou el servo fins a target_us de forma suau.
Incrementa/decrementa _INCREMENT_US cada mil·lisegon.
"""
global _current_servo_pos
if not (MIN_SERVO_US <= target_us <= MAX_SERVO_US):
return
while True:
if _current_servo_pos < target_us - _INCREMENT_US:
_current_servo_pos += _INCREMENT_US
elif _current_servo_pos > target_us + _INCREMENT_US:
_current_servo_pos -= _INCREMENT_US
else:
_current_servo_pos = target_us
_pi.set_servo_pulsewidth(SERVO_PWM, _current_servo_pos)
time.sleep(0.001)
if _current_servo_pos == target_us:
return
# ==================
# Sensor de color
# ==================
def _calc_colors_difference(measured: tuple, reference: dict) -> int:
"""Distància Manhattan entre el color mesurat i un color de referència."""
return (abs(measured[0] - reference["r"]) +
abs(measured[1] - reference["g"]) +
abs(measured[2] - reference["b"]))
def read_color_raw() -> tuple:
"""Retorna (r, g, b) en bytes 0-255 sense classificació. Útil per calibrar."""
try:
return _color_sensor.color_rgb_bytes
except Exception:
return (0, 0, 0)
def read_block_color() -> int:
"""
Llegeix el color del bloc des del sensor TCS34725.
Compara contra la taula de referència per distància Manhattan.
Retorna l'ID del color més proper, o BK si cap supera el llindar.
"""
MAX_DIFFERENCE = 15
try:
r, g, b = _color_sensor.color_rgb_bytes
except Exception:
return BK
min_difference = MAX_DIFFERENCE
min_diff_color = BK
for color_id, ref in enumerate(_COLORS):
diff = _calc_colors_difference((r, g, b), ref)
if diff < min_difference:
min_difference = diff
min_diff_color = color_id
return min_diff_color

273
raspi/eyes.py Normal file
View File

@@ -0,0 +1,273 @@
"""
eyes.py — Control de les matrius LED 8x8 RGB WS2811 (ulls del robot).
Equivalent a eyes.cpp del codi Arduino/ESP32.
FastLED → pigpio waveforms (GPIO26, qualsevol pin).
REQUISIT: iniciar el dimoni pigpio amb resolució d'1µs:
sudo pigpiod -s 1
Si s'inicia sense -s 1 (defecte 5µs), els LEDs no funcionaran correctament.
Si en el futur es fa una modificació hardware (GPIO26 → GPIO18 o GPIO21),
es pot substituir _send_ws2811() per rpi_ws281x sense canviar cap altra funció.
"""
import time
import threading
import pigpio
from pins import LED_DATA
# ==================
# Constants
# ==================
ROW_NUM = 8
COL_NUM = 8
NUM_LEDS = ROW_NUM * COL_NUM * 2 # 128 LEDs (2 matrius 8x8)
MAX_BR = 170 # Brillantor màxima del parpelleig (0–255)
MIN_BR = 80 # Brillantor mínima del parpelleig
# Colors predefinits (R, G, B)
WHITE = (255, 255, 255)
RED = (255, 0, 0)
GREEN = ( 0, 255, 0)
BLUE = ( 0, 0, 255)
YELLOW = (255, 200, 0)
ORANGE = (255, 80, 0)
PURPLE = (180, 0, 255)
CYAN = ( 0, 255, 255) # Color del mode gestos
BLACK = ( 0, 0, 0)
# ==================
# Formes dels ulls (índexs dels LEDs actius)
# ==================
class EyeShape:
"""Conjunt de LEDs que formen una expressió dels ulls."""
def __init__(self, leds: list):
self.leds = leds
self.len = len(leds)
EYES_OPEN = EyeShape([
102, 89, 38, 25, 106, 101, 90, 85, 42, 37, 26, 21,
107, 100, 91, 84, 43, 36, 27, 20, 108, 99, 92, 83,
44, 35, 28, 19, 109, 98, 93, 82, 45, 34, 29, 18,
97, 94, 33, 30
])
EYES_FW = EyeShape([
103, 88, 39, 24, 105, 102, 89, 86, 41, 38, 25, 22,
117, 106, 101, 90, 85, 74, 53, 42, 37, 26, 21, 10,
123, 116, 100, 91, 75, 68, 59, 52, 36, 27, 11, 4,
99, 92, 35, 28, 98, 93, 34, 29, 97, 94, 33, 30, 96,
95, 32, 31
])
EYES_DOWN = EyeShape([97, 94, 33, 30])
# Nova forma per al mode gestos: marc extern dels dos ulls (expressió "atenta")
EYES_GESTURE = EyeShape([
96, 97, 98, 99, 100, 101, 102, 103,
104, 111, 112, 119, 120, 127,
31, 32, 33, 34, 35, 36, 37, 38, 39,
0, 7, 8, 15, 16, 23
])
# ==================
# Estat global
# ==================
_pi: pigpio.pi = None
_leds = [[0, 0, 0] for _ in range(NUM_LEDS)]
_brightness = MAX_BR
_leds_lock = threading.Lock()
_update_stop = threading.Event()
_update_thread: threading.Thread = None
# Màscara GPIO per a les waveforms de pigpio
_GPIO_MASK: int = 0
# ==================
# WS2811 via pigpio waveforms
# ==================
def _send_ws2811(data: bytes):
"""
Envia dades RGB als LEDs WS2811 via pigpio waveforms.
Ordre de color: GRB (igual que FastLED amb WS2811, GRB).
Timing a 1µs de resolució (requereix sudo pigpiod -s 1):
- Bit 0: 1µs HIGH + 2µs LOW (spec: 0.5µs + 2.0µs)
- Bit 1: 2µs HIGH + 1µs LOW (spec: 1.2µs + 1.3µs)
- Reset: 80µs LOW
"""
pulses = []
for byte_val in data:
for bit in range(7, -1, -1):
if byte_val & (1 << bit):
pulses.append(pigpio.pulse(_GPIO_MASK, 0, 2))
pulses.append(pigpio.pulse(0, _GPIO_MASK, 1))
else:
pulses.append(pigpio.pulse(_GPIO_MASK, 0, 1))
pulses.append(pigpio.pulse(0, _GPIO_MASK, 2))
pulses.append(pigpio.pulse(0, _GPIO_MASK, 80)) # reset
_pi.wave_add_new()
_pi.wave_add_generic(pulses)
wid = _pi.wave_create()
if wid >= 0:
_pi.wave_send_once(wid)
while _pi.wave_tx_busy():
pass
_pi.wave_delete(wid)
def _eyes_show(brightness: int):
"""Renderitza l'estat actual de _leds amb la brillantor indicada."""
data = bytearray(NUM_LEDS * 3)
scale = brightness / 255
for i, (r, g, b) in enumerate(_leds):
data[i * 3 + 0] = int(g * scale) # WS2811 GRB: primer G
data[i * 3 + 1] = int(r * scale)
data[i * 3 + 2] = int(b * scale)
_send_ws2811(bytes(data))
# ==================
# Setup i cleanup
# ==================
def eyes_setup(pi: pigpio.pi):
"""Inicialitza el GPIO i arrenca el thread de parpelleig."""
global _pi, _GPIO_MASK, _update_thread
_pi = pi
_GPIO_MASK = 1 << LED_DATA
pi.set_mode(LED_DATA, pigpio.OUTPUT)
pi.write(LED_DATA, 0)
_update_stop.clear()
_update_thread = threading.Thread(
target=_task_update_leds, daemon=True, name="eyes"
)
_update_thread.start()
def eyes_cleanup():
"""Atura el thread de parpelleig i apaga els LEDs."""
_update_stop.set()
if _update_thread:
_update_thread.join(timeout=1.0)
with _leds_lock:
for i in range(NUM_LEDS):
_leds[i] = [0, 0, 0]
_eyes_show(255)
# ==================
# Thread de parpelleig (equivalent a task_update_leds del FreeRTOS)
# ==================
def _task_update_leds():
"""
Bucle continu que fa respirar la brillantor dels ulls.
Equivalent a task_update_leds() del FreeRTOS.
"""
global _brightness
going_up = False # Al C++ comença a MAX_BR i baixa
_brightness = MAX_BR
while not _update_stop.is_set():
if going_up:
if _brightness < MAX_BR:
_brightness += 2
else:
going_up = False
else:
if _brightness > MIN_BR:
_brightness -= 2
else:
going_up = True
with _leds_lock:
_eyes_show(_brightness)
time.sleep(0.05)
# ==================
# Animacions (equivalent a les funcions de eyes.cpp)
# ==================
def eyes_turn_off():
"""Apaga tots els LEDs amb un fos progressiu."""
for _ in range(50):
with _leds_lock:
for i in range(NUM_LEDS):
r, g, b = _leds[i]
_leds[i] = [int(r * 245 / 255),
int(g * 245 / 255),
int(b * 245 / 255)]
time.sleep(0.01)
with _leds_lock:
for i in range(NUM_LEDS):
_leds[i] = [0, 0, 0]
def eyes_turn_on(shape: EyeShape, color: tuple,
repeat: int = 1, forward: bool = True):
"""
Encén els LEDs d'una forma un per un, amb animació.
shape: forma a dibuixar (EYES_OPEN, EYES_FW, EYES_DOWN…)
color: color RGB com a tupla (r, g, b)
repeat: nombre de vegades que es repeteix l'animació
forward: True = ordre normal, False = ordre invers
"""
r, g, b = color
for rep in range(repeat):
eyes_turn_off()
for i in range(shape.len):
idx = shape.leds[i if forward else (shape.len - 1 - i)]
with _leds_lock:
_leds[idx] = [r, g, b]
time.sleep(0.008)
if rep < repeat - 1:
for i in range(shape.len):
idx = shape.leds[i if forward else (shape.len - 1 - i)]
with _leds_lock:
_leds[idx] = [0, 0, 0]
time.sleep(0.008)
# ==================
# Animacions noves — mode gestos (TFG)
# ==================
def eyes_gesture_mode_on():
"""
Animació d'activació del mode gestos.
Parpelleig doble en cian per indicar que el robot escolta gestos.
"""
eyes_turn_on(EYES_OPEN, CYAN, repeat=2)
def eyes_gesture_mode_off():
"""
Animació de desactivació del mode gestos.
Torna als ulls oberts en blanc.
"""
eyes_turn_off()
eyes_turn_on(EYES_OPEN, WHITE)
def eyes_listening():
"""
Expressió "escoltant": marc extern dels ulls en cian.
Es mostra mentre el robot espera un gest.
"""
eyes_turn_on(EYES_GESTURE, CYAN)

238
raspi/gesture.py Normal file
View File

@@ -0,0 +1,238 @@
"""
gesture.py — Lectura del sensor de gestos PAJ7620U2 via I2C raw (smbus2).
Equivalent a gesture.cpp del codi Arduino/ESP32.
No hi ha pin INT disponible al PCB → polling cada 50ms en un thread.
Bus I2C 3 (GPIO2=SDA, GPIO1=SCL, compartit amb VL53L0X i ADS1115).
Llibreria C++ equivalent: RevEng_PAJ7620 (Aaron S. Crandall)
"""
import time
import threading
import smbus2
# ==================
# IDs de gest
# ==================
GS_NONE = 0
GS_FORWARD = 1
GS_LEFT = 2
GS_RIGHT = 3
GS_UP = 4
GS_DOWN = 5
GS_CLOCKWISE = 6
GS_ANTICLOCKWISE = 7
GS_WAVE = 8
# Àlies per al quibot.py
GS_CW = GS_CLOCKWISE
GS_CCW = GS_ANTICLOCKWISE
# ==================
# Registres PAJ7620U2
# ==================
_PAJ7620_ADDR = 0x73
_REG_BANK_SEL = 0xEF # Registre de selecció de banc (0x00=banc0, 0x01=banc1)
_REG_PART_ID_LSB = 0x00 # Ha de retornar 0x20
_REG_PART_ID_MSB = 0x01 # Ha de retornar 0x76
_REG_GESTURE_0 = 0x43 # Bits 0–7: Right, Left, Up, Down, Forward, Backward, CW, CCW
_REG_GESTURE_1 = 0x44 # Bit 0: Wave
# Bits de gest al registre 0x43
_BIT_RIGHT = 0x01
_BIT_LEFT = 0x02
_BIT_UP = 0x04
_BIT_DOWN = 0x08
_BIT_FORWARD = 0x10
_BIT_BACKWARD = 0x20
_BIT_CW = 0x40
_BIT_CCW = 0x80
# Bit de gest al registre 0x44
_BIT_WAVE = 0x01
# ==================
# Seqüències d'inicialització
# ==================
_INIT_BANK0 = [
(0x32, 0x29), (0x33, 0x01), (0x34, 0x00), (0x35, 0x01), (0x36, 0x00),
(0x37, 0x07), (0x38, 0x17), (0x39, 0x06), (0x3A, 0x12), (0x3F, 0x00),
(0x40, 0x02), (0x41, 0xFF), (0x42, 0x01), (0x46, 0x2D), (0x47, 0x0F),
(0x48, 0x3C), (0x49, 0x00), (0x4A, 0x1E), (0x4B, 0x00), (0x4C, 0x20),
(0x4D, 0x00), (0x4E, 0x1A), (0x4F, 0x14), (0x50, 0x00), (0x51, 0x10),
(0x52, 0x00), (0x5C, 0x02), (0x5D, 0x00), (0x5E, 0x10), (0x5F, 0x3F),
(0x60, 0x27), (0x61, 0x28), (0x62, 0x00), (0x63, 0x03), (0x64, 0xF7),
(0x65, 0x03), (0x66, 0xD9), (0x67, 0x03), (0x68, 0x01), (0x69, 0xC8),
(0x6A, 0x40), (0x6D, 0x04), (0x6E, 0x00), (0x6F, 0x00), (0x70, 0x80),
(0x71, 0x00), (0x72, 0x00), (0x73, 0x00), (0x74, 0xF0), (0x75, 0x00),
(0x80, 0x42), (0x81, 0x44), (0x82, 0x04), (0x83, 0x20), (0x84, 0x20),
(0x85, 0x00), (0x86, 0x10), (0x87, 0x00), (0x88, 0x05), (0x89, 0x18),
(0x8A, 0x10), (0x8B, 0x01), (0x8C, 0x37), (0x8D, 0x00), (0x8E, 0xF0),
(0x8F, 0x81), (0x90, 0x06), (0x91, 0x06), (0x92, 0x1E), (0x93, 0x0D),
(0x94, 0x0A), (0x95, 0x0A), (0x96, 0x0C), (0x97, 0x05), (0x98, 0x0A),
(0x99, 0x41), (0x9A, 0x14), (0x9B, 0x0A), (0x9C, 0x3F), (0x9D, 0x33),
(0x9E, 0xAE), (0x9F, 0xF9), (0xA0, 0x48), (0xA1, 0x13), (0xA2, 0x10),
(0xA3, 0x08), (0xA4, 0x30), (0xA5, 0x19), (0xA6, 0x10), (0xA7, 0x08),
(0xA8, 0x24), (0xA9, 0x04), (0xAA, 0x1E), (0xAB, 0x1E), (0xCC, 0x19),
(0xCD, 0x0B), (0xCE, 0x13), (0xCF, 0x64), (0xD0, 0x21), (0xD1, 0x0F),
(0xD2, 0x88), (0xE0, 0x01), (0xE1, 0x04), (0xE2, 0x41), (0xE3, 0xD6),
(0xE4, 0x00), (0xE5, 0x0C), (0xE6, 0x0A), (0xE7, 0x00), (0xE8, 0x00),
(0xE9, 0x00), (0xEE, 0x07),
]
_INIT_BANK1 = [
(0x00, 0x1E), (0x01, 0x1E), (0x02, 0x0F), (0x03, 0x10), (0x04, 0x02),
(0x05, 0x00), (0x06, 0xB0), (0x07, 0x04), (0x08, 0x0D), (0x09, 0x0E),
(0x0A, 0x9C), (0x0B, 0x04), (0x0C, 0x05), (0x0D, 0x0F), (0x0E, 0x02),
(0x0F, 0x12), (0x10, 0x02), (0x11, 0x02), (0x12, 0x00), (0x13, 0x01),
(0x14, 0x05), (0x15, 0x07), (0x16, 0x05), (0x17, 0x07), (0x18, 0x01),
(0x19, 0x04), (0x1A, 0x05), (0x1B, 0x0C), (0x1C, 0x2A), (0x1D, 0x01),
(0x1E, 0x00), (0x21, 0x00), (0x22, 0x00), (0x23, 0x00), (0x25, 0x01),
(0x26, 0x00), (0x27, 0x39), (0x28, 0x7F), (0x29, 0x08), (0x30, 0x03),
(0x31, 0x00), (0x32, 0x1A), (0x33, 0x1A), (0x34, 0x07), (0x35, 0x07),
(0x36, 0x01), (0x37, 0xFF), (0x38, 0x36), (0x39, 0x07), (0x3A, 0x00),
(0x3E, 0xFF), (0x3F, 0x00), (0x40, 0x77), (0x41, 0x40), (0x42, 0x00),
(0x43, 0x30), (0x44, 0xA0), (0x45, 0x5C), (0x46, 0x00), (0x47, 0x00),
(0x48, 0x58), (0x4A, 0x1E), (0x4B, 0x1E), (0x4C, 0x00), (0x4D, 0x00),
(0x4E, 0xA0), (0x4F, 0x80), (0x50, 0x00), (0x51, 0x00), (0x52, 0x00),
(0x53, 0x00), (0x54, 0x00), (0x57, 0x80), (0x59, 0x10), (0x5A, 0x08),
(0x5B, 0x94), (0x5C, 0xE8), (0x5D, 0x08), (0x5E, 0x3D), (0x5F, 0x99),
(0x60, 0x45), (0x61, 0x40), (0x63, 0x2D), (0x64, 0x02), (0x65, 0x96),
(0x66, 0x00), (0x67, 0x97), (0x68, 0x01), (0x69, 0xCD), (0x6A, 0x01),
(0x6B, 0xB0), (0x6C, 0x04), (0x6D, 0x2C), (0x6E, 0x01), (0x6F, 0x32),
(0x71, 0x00), (0x72, 0x01), (0x73, 0x35), (0x74, 0x00), (0x75, 0x33),
(0x76, 0x31), (0x77, 0x01), (0x7C, 0x84), (0x7D, 0x03), (0x7E, 0x01),
]
# ==================
# Estat global
# ==================
_bus: smbus2.SMBus = None
_gesture: int = GS_NONE
_gesture_lock = threading.Lock()
_poll_stop = threading.Event()
# ==================
# Helpers I2C
# ==================
def _select_bank(bank: int):
_bus.write_byte_data(_PAJ7620_ADDR, _REG_BANK_SEL, bank)
def _write(reg: int, val: int):
_bus.write_byte_data(_PAJ7620_ADDR, reg, val)
def _read(reg: int) -> int:
return _bus.read_byte_data(_PAJ7620_ADDR, reg)
# ==================
# Setup
# ==================
def gesture_setup():
"""
Inicialitza el PAJ7620U2 via I2C raw (smbus2, bus 3).
Arrenca el thread de polling (equivalent al ISR del C++).
"""
global _bus
_bus = smbus2.SMBus(3)
# Desperta el sensor (primer accés I2C)
try:
_bus.write_byte(_PAJ7620_ADDR, 0)
except Exception:
pass
time.sleep(0.001) # 700µs d'espera per wake-up
# Verifica ID del dispositiu
_select_bank(0)
id_lsb = _read(_REG_PART_ID_LSB)
id_msb = _read(_REG_PART_ID_MSB)
if id_lsb != 0x20 or id_msb != 0x76:
print(f"ERROR: PAJ7620U2 NOT FOUND (ID: {id_msb:02X}{id_lsb:02X})")
else:
print("Gesture sensor init OK")
# Escriu registres d'inicialització (banc 0)
_select_bank(0)
for reg, val in _INIT_BANK0:
_write(reg, val)
# Escriu registres d'inicialització (banc 1)
_select_bank(1)
for reg, val in _INIT_BANK1:
_write(reg, val)
# Torna al banc 0 per a la lectura de gestos
_select_bank(0)
# Arrenca el thread de polling
_poll_stop.clear()
threading.Thread(target=_poll_loop, daemon=True, name="gesture").start()
def gesture_cleanup():
"""Atura el polling i tanca el bus I2C."""
_poll_stop.set()
if _bus:
_bus.close()
# ==================
# Thread de polling (equivalent al ISR + flag del C++)
# ==================
def _poll_loop():
"""
Llegeix els registres de gest cada 50ms.
Equivalent a on_gesture_interrupt() + gesture_available flag del C++.
"""
global _gesture
while not _poll_stop.is_set():
try:
g0 = _read(_REG_GESTURE_0)
g1 = _read(_REG_GESTURE_1)
detected = GS_NONE
if g0 & _BIT_FORWARD: detected = GS_FORWARD
elif g0 & _BIT_LEFT: detected = GS_LEFT
elif g0 & _BIT_RIGHT: detected = GS_RIGHT
elif g0 & _BIT_UP: detected = GS_UP
elif g0 & _BIT_DOWN: detected = GS_DOWN
elif g0 & _BIT_CW: detected = GS_CLOCKWISE
elif g0 & _BIT_CCW: detected = GS_ANTICLOCKWISE
elif g1 & _BIT_WAVE: detected = GS_WAVE
if detected != GS_NONE:
with _gesture_lock:
_gesture = detected
except Exception:
pass # error I2C puntual → ignora i continua
time.sleep(0.05) # 50ms de polling (20Hz)
# ==================
# Lectura de gest
# ==================
def read_gesture() -> int:
"""
Retorna l'últim gest detectat i el reseteja a GS_NONE.
No bloquejant — equivalent a read_gesture() del C++.
"""
global _gesture
with _gesture_lock:
gest = _gesture
_gesture = GS_NONE
return gest

210
raspi/main.py Normal file
View File

@@ -0,0 +1,210 @@
from fastapi import FastAPI, File, Form, UploadFile, HTTPException, Query
from fastapi.middleware.cors import CORSMiddleware
import subprocess
import threading
import time
import os
import json
import uuid
import hashlib
from pathlib import Path
from pydantic import BaseModel
import RPi.GPIO as GPIO
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["*"],
allow_headers=["*"],
)
INCOMING_DIR = Path("/tmp/quibot-audio/incoming")
LOCKED_DIR = Path("/tmp/quibot-audio/locked")
PROCESSED_DIR = Path("/tmp/quibot-audio/processed")
INCOMING_DIR.mkdir(parents=True, exist_ok=True)
LOCKED_DIR.mkdir(parents=True, exist_ok=True)
PROCESSED_DIR.mkdir(parents=True, exist_ok=True)
# -------------------------
# GPIO SETUP
# -------------------------
STEP = 23
DIR = 24
EN = 25
GPIO.setmode(GPIO.BCM)
GPIO.setup(STEP, GPIO.OUT)
GPIO.setup(DIR, GPIO.OUT)
GPIO.setup(EN, GPIO.OUT)
GPIO.output(EN, GPIO.LOW)
motor_thread = None
def step_motor(steps, direction, delay=0.001):
GPIO.output(DIR, direction)
for _ in range(steps):
GPIO.output(STEP, GPIO.HIGH)
time.sleep(delay)
GPIO.output(STEP, GPIO.LOW)
time.sleep(delay)
def motor_step(dir):
dir_pin = GPIO.HIGH if dir == "forward" else GPIO.LOW
time.sleep(0.02) # small delay before starting
print("Motor running...")
step_motor(200, dir_pin, 0.001)
# -------------------------
# SAFE COMMAND WHITELIST
# -------------------------
COMMANDS = {
"restart_nginx": ["sudo", "systemctl", "restart", "nginx"],
"uptime": ["uptime"],
"update": ["sudo", "apt", "update"]
}
# -------------------------
# API ENDPOINTS
# -------------------------
@app.post("/run")
def run_task(task: str, token: str):
if token != "MY_SECRET_TOKEN":
raise HTTPException(status_code=403, detail="Unauthorized")
if task not in COMMANDS:
raise HTTPException(status_code=400, detail="Invalid task")
try:
result = subprocess.check_output(COMMANDS[task], text=True)
return {"output": result}
except subprocess.CalledProcessError as e:
return {"error": e.output}
@app.post("/motor/step/forward")
def start_motor(token: str):
global motor_thread
if token != "MY_SECRET_TOKEN":
raise HTTPException(status_code=403, detail="Unauthorized")
motor_thread = threading.Thread(target=motor_step, args=("forward",), daemon=True)
motor_thread.start()
return {"status": "motor started"}
@app.post("/motor/step/backwards")
def start_motor(token: str):
global motor_thread
if token != "MY_SECRET_TOKEN":
raise HTTPException(status_code=403, detail="Unauthorized")
motor_thread = threading.Thread(target=motor_step, args=("backwards",), daemon=True)
motor_thread.start()
return {"status": "motor started"}
@app.post("/motor/stop")
def stop_motor(token: str):
if token != "MY_SECRET_TOKEN":
raise HTTPException(status_code=403, detail="Unauthorized")
GPIO.output(EN, GPIO.HIGH) # disable driver
return {"status": "motor stopped"}
@app.post("/audio/upload")
async def upload_audio(file: UploadFile = File(...), format: str = "wav"):
raw_content = await file.read()
checksum = hashlib.sha256(raw_content).hexdigest()[:16]
filename = f"{checksum[:10]}-{uuid.uuid4().hex[:8]}.wav"
filepath = INCOMING_DIR / filename
filepath.write_bytes(raw_content)
return {"status": "received", "filename": str(filepath), "lock_url": f"/audio/lock/{filepath.name}"}
@app.get("/audio/incoming")
def list_incoming():
files = []
for f in sorted(INCOMING_DIR.iterdir()):
meta = f.stat()
files.append({
"filename": f.name,
"size_bytes": meta.st_size,
"modified_iso": time.ctime(meta.st_mtime),
})
return {"count": len(files), "files": files}
@app.post("/audio/lock/{filename}")
def lock_audio(filename: str):
src = INCOMING_DIR / filename
dst = LOCKED_DIR / filename
if not src.exists():
raise HTTPException(status_code=404, detail=f"File {filename} not found")
if dst.exists():
return {"status": "already_locked", "filename": filename}
os.rename(str(src), str(dst))
return {"status": "locked", "filename": filename}
@app.post("/audio/unlock/{filename}")
def unlock_audio(filename: str):
src = LOCKED_DIR / filename
dst = INCOMING_DIR / filename
if not src.exists():
raise HTTPException(status_code=404, detail=f"File {filename} not found")
os.rename(str(src), str(dst))
return {"status": "unlocked", "filename": filename}
@app.post("/audio/cancel/{filename}")
def cancel_audio(filename: str):
src = LOCKED_DIR / filename
dst = INCOMING_DIR / filename
if not src.exists():
raise HTTPException(status_code=404, detail=f"File {filename} not found")
os.rename(str(src), str(dst))
return {"status": "cancelled", "filename": filename}
@app.post("/audio/process/{filename}")
def process_audio(filename: str):
locked = LOCKED_DIR / filename
processed = PROCESSED_DIR / filename
if not locked.exists():
raise HTTPException(status_code=404, detail=f"File {filename} not found")
os.rename(str(locked), str(processed))
return {"status": "processed", "filename": filename}
@app.on_event("shutdown")
def shutdown():
global motor_running
motor_running = False
GPIO.output(EN, GPIO.HIGH)
GPIO.cleanup()

619
raspi/motion.py Normal file
View File

@@ -0,0 +1,619 @@
"""
motion.py — Control de motors (rodes, braços, xeringa), sensor de distància
VL53L0X i seguidor de línia TCRT5000 via ADS1115.
Equivalent a motion.cpp del codi Arduino/ESP32.
"""
import math
import time
import threading
import pigpio
import adafruit_extended_bus
import adafruit_vl53l0x
import adafruit_ads1x15.ads1115 as ADS
from adafruit_ads1x15.analog_in import AnalogIn
from pins import (
STEP_R_W, DIR_R_W, STEP_L_W, DIR_L_W, EN_W,
STEP_R_A, DIR_R_A, STEP_L_A, DIR_L_A, EN_A,
STEP_SY, DIR_SY, EN_SERVO,
END_SY, END_RA, END_LA,
)
# ==================
# Constants
# ==================
CW = True
CCW = False
ON = 0 # A4988/TB6600: enable actiu en LOW
OFF = 1
TAKE = True
LEAVE = False
# Tipus de resposta del seguidor de línia
CLEAR = 0
CROSSING = 1
OBJECT = 2
# Noms de tasques (equivalent a les constants string del C++)
MOVE_TO_CROSSING = "Move to crossing"
TURN_90_CW = "Turn 90 CW"
TURN_90_CCW = "Turn 90 CCW"
MOVE_TO_OBJECT = "Move to object"
TAKE_SOMETHING = "Take something"
LEAVE_SOMETHING = "Leave something"
DO_NOTHING = "Do nothing"
# Paràmetres de moviment
WHEELS_MAX_SPEED = 130.0 # steps/s
WHEELS_ACCEL = 190.0 # steps/s²
ARMS_MAX_SPEED = 250.0 # steps/s
ARMS_ACCEL = 125.0 # steps/s²
SYRINGE_MAX_SPEED = 800.0 # steps/s
SYRINGE_ACCEL = 500.0 # steps/s²
WHEEL_MECH_REDUCTION = 5
WHEEL_STEPS_PER_REVOLUTION = 200 * WHEEL_MECH_REDUCTION # 1000 passos/volta de roda
MM_TO_CROSSING_CENTER = 62 # mm des del creuament detectat fins al centre
MM_TO_OBJECT = 20 # mm addicionals un cop detectat l'objecte
# Llindar de negre per ADS1115 (GAIN_ONE ±4.096V, single-ended 0–26400 per a 3.3V).
# Equivalent a 1500/4095 de l'ESP32 de 12 bits → ~9700 en ADS1115.
BLACK_THRESHOLD = 9700
# Llindar d'error de rotació (equivalent a 20/4095 de l'ESP32 → ~130 en ADS1115).
ROTATION_ERROR_THRESHOLD = 130
# Posicions dels braços en passos des del home
ARM_LOWER_POSITION = 120
ARM_L_UPPER_POSITION = 900
ARM_R_UPPER_POSITION = 550
# Xeringa: 10 rev * 200 passos/rev * microstepping x4 = 8000 passos estesa del tot
_SY_FULL_EXTENDED_STEPS = 10 * 200 * 4
LINE_FOLLOWER_FREQ = 100 # Hz
LINE_FOLLOWER_PERIOD = 1.0 / LINE_FOLLOWER_FREQ # s
# ==================
# Classe Stepper
# ==================
class Stepper:
"""
Motor pas a pas en mode DRIVER (STEP/DIR).
Equivalent a AccelStepper(DRIVER, step_pin, dir_pin).
Genera polsos STEP via pigpio.gpio_trigger().
"""
PULSE_US = 10 # Amplada del pols STEP en µs (A4988 requereix ≥1µs)
def __init__(self, pi: pigpio.pi, step_pin: int, dir_pin: int):
self._pi = pi
self._step_pin = step_pin
self._dir_pin = dir_pin
self._pos = 0 # posició actual (passos)
self._target = 0 # posició objectiu
self._speed = 0.0 # velocitat actual (passos/s, signada)
self._max_speed = 1.0
self._accel = 1.0
self._last_step_us = self._now_us()
self._step_interval_us = 0 # 0 = aturat
pi.set_mode(step_pin, pigpio.OUTPUT)
pi.set_mode(dir_pin, pigpio.OUTPUT)
pi.write(step_pin, 0)
pi.write(dir_pin, 0)
@staticmethod
def _now_us() -> int:
return time.monotonic_ns() // 1000
def set_max_speed(self, speed: float):
self._max_speed = abs(speed)
def set_acceleration(self, accel: float):
self._accel = abs(accel)
def move_to(self, position: int):
self._target = int(position)
def move(self, relative: int):
self._target = self._pos + int(relative)
def set_current_position(self, pos: int):
self._pos = int(pos)
self._target = int(pos)
self._speed = 0.0
self._step_interval_us = 0
def current_position(self) -> int:
return self._pos
def distance_to_go(self) -> int:
return self._target - self._pos
def is_running(self) -> bool:
return self._target != self._pos
def stop(self):
self._target = self._pos
self._speed = 0.0
self._step_interval_us = 0
def set_speed(self, speed: float):
"""Estableix velocitat constant per a run_speed()."""
self._speed = float(speed)
self._step_interval_us = int(1_000_000 / abs(speed)) if speed != 0.0 else 0
def _do_step(self, direction: int):
self._pi.write(self._dir_pin, 1 if direction > 0 else 0)
self._pi.gpio_trigger(self._step_pin, self.PULSE_US, 1)
self._pos += direction
def run_speed(self) -> bool:
"""Fa un pas a velocitat constant. No bloquejant — cridar des del bucle de steppers."""
if self._step_interval_us == 0:
return False
now = self._now_us()
if now - self._last_step_us >= self._step_interval_us:
self._do_step(1 if self._speed > 0 else -1)
self._last_step_us = now
return True
return False
def run(self) -> bool:
"""
Fa un pas cap a _target amb acceleració/desacceleració.
No bloquejant — cridar des del bucle de steppers.
Implementa l'algorisme de AccelStepper: Δv = accel / v per pas.
"""
dtg = self.distance_to_go()
if dtg == 0:
self._speed = 0.0
self._step_interval_us = 0
return False
abs_speed = abs(self._speed)
if abs_speed < 1.0:
abs_speed = math.sqrt(self._accel / 2.0) # velocitat inicial AccelStepper
now = self._now_us()
if now - self._last_step_us < int(1_000_000 / abs_speed):
return False # no és hora del proper pas
# Actualitza la velocitat per al proper pas
direction = 1 if dtg > 0 else -1
stop_dist = (abs_speed ** 2) / (2.0 * self._accel) if self._accel > 0 else 0
if abs(dtg) <= max(stop_dist, 1):
new_speed = abs_speed - (self._accel / abs_speed)
new_speed = max(new_speed, 1.0)
else:
new_speed = abs_speed + (self._accel / abs_speed)
new_speed = min(new_speed, self._max_speed)
self._speed = new_speed * direction
self._do_step(direction)
self._last_step_us = now
return True
# ==================
# Instàncies globals
# ==================
_pi: pigpio.pi = None
_i2c = None
_dist_sensor = None
_ads = None
_chan_r: AnalogIn = None
_chan_l: AnalogIn = None
wheel_R: Stepper = None
wheel_L: Stepper = None
arm_R: Stepper = None
arm_L: Stepper = None
syringe: Stepper = None
wheels_speed_mode: bool = False # True → run_speed(), False → run() (posició)
_stepper_stop = threading.Event()
_stepper_thread: threading.Thread = None
# ==================
# Setup i cleanup
# ==================
def motion_setup_steppers(pi: pigpio.pi):
"""
Inicialitza GPIOs i steppers sense els sensors I2C.
Útil per a tests de motors quan VL53L0X/ADS1115 no estan connectats.
"""
global _pi, wheel_R, wheel_L, arm_R, arm_L, syringe, _stepper_thread
_pi = pi
for pin in (EN_W, EN_A, EN_SERVO):
pi.set_mode(pin, pigpio.OUTPUT)
for pin in (END_LA, END_RA, END_SY):
pi.set_mode(pin, pigpio.INPUT)
pi.set_pull_up_down(pin, pigpio.PUD_UP) # Hall actiu-baix, pull-up intern
enable_arms(OFF)
enable_wheels(OFF)
wheel_R = Stepper(pi, STEP_R_W, DIR_R_W)
wheel_L = Stepper(pi, STEP_L_W, DIR_L_W)
arm_R = Stepper(pi, STEP_R_A, DIR_R_A)
arm_L = Stepper(pi, STEP_L_A, DIR_L_A)
syringe = Stepper(pi, STEP_SY, DIR_SY)
wheel_R.set_max_speed(WHEELS_MAX_SPEED); wheel_R.set_acceleration(WHEELS_ACCEL)
wheel_L.set_max_speed(WHEELS_MAX_SPEED); wheel_L.set_acceleration(WHEELS_ACCEL)
arm_R.set_max_speed(ARMS_MAX_SPEED); arm_R.set_acceleration(ARMS_ACCEL)
arm_L.set_max_speed(ARMS_MAX_SPEED); arm_L.set_acceleration(ARMS_ACCEL)
syringe.set_max_speed(SYRINGE_MAX_SPEED); syringe.set_acceleration(SYRINGE_ACCEL)
_stepper_stop.clear()
_stepper_thread = threading.Thread(target=_stepper_loop, daemon=True, name="steppers")
_stepper_thread.start()
def motion_setup_sensors(pi: pigpio.pi):
"""
Inicialitza únicament els sensors I2C (VL53L0X, ADS1115). Sense steppers.
Útil per a tests de sensors quan els motors no estan connectats.
"""
global _pi, _i2c, _dist_sensor, _ads, _chan_r, _chan_l
_pi = pi
_i2c = adafruit_extended_bus.ExtendedI2C(3)
_dist_sensor = adafruit_vl53l0x.VL53L0X(_i2c)
_ads = ADS.ADS1115(_i2c)
_chan_r = AnalogIn(_ads, ADS.P0)
_chan_l = AnalogIn(_ads, ADS.P1)
def motion_setup(pi: pigpio.pi):
"""
Inicialitza GPIOs, steppers, sensor de distància VL53L0X,
ADC ADS1115 per als sensors de línia, i arrenca el bucle de steppers.
Requereix /boot/config.txt: dtoverlay=i2c-gpio,bus=3,i2c_gpio_sda=2,i2c_gpio_scl=1
"""
motion_setup_steppers(pi)
motion_setup_sensors(pi)
def motion_cleanup():
"""Atura el bucle de steppers i desactiva tots els motors."""
_stepper_stop.set()
if _stepper_thread:
_stepper_thread.join(timeout=1.0)
enable_wheels(OFF)
enable_arms(OFF)
enable_syringe(OFF)
# ==================
# Enable / disable
# ==================
def enable_wheels(state: bool):
_pi.write(EN_W, state)
def enable_arms(state: bool):
_pi.write(EN_A, state)
def enable_syringe(state: bool):
_pi.write(EN_SERVO, state)
def is_endstop_detecting(pin: int) -> bool:
return not _pi.read(pin) # efecte Hall actiu en baix
# ==================
# Helpers de moviment
# ==================
def mm_to_steps(mm: int) -> int:
# Perímetre roda Ø152mm = 2·π·76 ≈ 477mm
return (mm * WHEEL_STEPS_PER_REVOLUTION * 2) // 1000
def wheels_set_position(position: int = 0):
wheel_L.set_current_position(position)
wheel_R.set_current_position(position)
def wheels_set_speed(speed: float, rotate: bool = False, direction: bool = CW):
if rotate:
wheel_L.set_speed( speed if direction == CW else -speed)
wheel_R.set_speed(-speed if direction == CW else speed)
else:
wheel_L.set_speed(speed)
wheel_R.set_speed(speed)
def move_arms_to(position: int):
arm_L.move_to(position)
arm_R.move_to(position)
while arm_L.distance_to_go() != 0 and arm_R.distance_to_go() != 0:
time.sleep(0.1)
def move_arms_up():
arm_L.move_to(ARM_L_UPPER_POSITION)
arm_R.move_to(ARM_R_UPPER_POSITION)
while arm_L.distance_to_go() != 0 and arm_R.distance_to_go() != 0:
time.sleep(0.1)
def move_wheels_to(position: int, invert: bool = False):
wheel_L.move_to(position)
wheel_R.move_to(-position if invert else position)
while wheel_L.is_running() and wheel_R.is_running():
time.sleep(0.1)
# ==================
# Homing
# ==================
def arms_home():
"""Cicle de homing dels braços (bloquejant). El bucle de steppers fa el moviment."""
enable_arms(ON)
arm_L.move(-1250)
arm_R.move(-1250)
while True:
if is_endstop_detecting(END_LA):
arm_L.stop()
if is_endstop_detecting(END_RA):
arm_R.stop()
l_done = is_endstop_detecting(END_LA) or arm_L.distance_to_go() == 0
r_done = is_endstop_detecting(END_RA) or arm_R.distance_to_go() == 0
if l_done and r_done:
break
time.sleep(0.005)
arm_L.set_current_position(0)
arm_R.set_current_position(0)
arm_L.move(ARM_L_UPPER_POSITION)
arm_R.move(ARM_R_UPPER_POSITION)
while arm_L.distance_to_go() != 0 or arm_R.distance_to_go() != 0:
time.sleep(0.01)
def syringe_home():
"""Cicle de homing de la xeringa (bloquejant). El bucle de steppers fa el moviment."""
enable_syringe(ON)
syringe.move(-11000)
while not is_endstop_detecting(END_SY):
if syringe.distance_to_go() == 0:
break
time.sleep(0.005)
syringe.stop()
syringe.set_current_position(0)
enable_syringe(OFF)
# ==================
# Sensor de distància
# ==================
def distance_to_object() -> int:
"""Retorna la distància en mm a l'objecte més proper. 65535 si fora de rang."""
try:
return _dist_sensor.range
except Exception:
return 65535
# ==================
# Seguidor de línia
# ==================
def _read_line_sensors() -> tuple:
"""Llegeix sensors de línia via ADS1115. Retorna (dreta, esquerra), 0–32767."""
return _chan_r.value, _chan_l.value
def compute_new_speed(speed: float) -> float:
accel = WHEELS_ACCEL / LINE_FOLLOWER_FREQ
return min(speed + accel, WHEELS_MAX_SPEED)
def follow_line_loop(speed: float, forward: bool = True) -> int:
"""
Seguidor de línia no bloquejant. Retorna CLEAR, CROSSING o OBJECT.
Si CLEAR, aplica correcció proporcional a les velocitats de les rodes.
"""
distance_threshold = 50 # mm
lf_r, lf_l = _read_line_sensors()
if lf_l > BLACK_THRESHOLD and lf_r > BLACK_THRESHOLD and forward:
return CROSSING
elif distance_to_object() < distance_threshold and forward:
return OBJECT
else:
p_factor = 5
error = lf_r - lf_l
correction_r = error // p_factor
correction_l = -correction_r
wheel_L.set_speed((speed + correction_r) if forward else (-speed + correction_r))
wheel_R.set_speed((speed + correction_l) if forward else (-speed + correction_l))
return CLEAR
def run_to_crossing_center(speed: float) -> float:
"""
Avança per centrar el robot sobre el creuament detectat.
Retorna la nova velocitat (reduïda a la meitat).
"""
wheels_set_position(0 - mm_to_steps(MM_TO_CROSSING_CENTER))
while follow_line_loop(speed) == CROSSING:
wheels_set_speed(speed)
time.sleep(0.1)
speed /= 2
while wheel_L.current_position() < 0 and wheel_R.current_position() < 0:
wheels_set_speed(speed)
time.sleep(0.1)
wheels_set_speed(0)
return speed
# ==================
# Bucle de steppers (thread permanent)
# ==================
def _stepper_loop():
"""
Thread que genera els polsos STEP per a tots els motors.
Equivalent a task_update_steppers() del FreeRTOS.
S'inicia automàticament a motion_setup().
"""
while not _stepper_stop.is_set():
if wheels_speed_mode:
wheel_R.run_speed()
wheel_L.run_speed()
else:
wheel_R.run()
wheel_L.run()
arm_R.run()
arm_L.run()
syringe.run()
# ==================
# Tasques (FreeRTOS tasks → funcions bloquejants, cridades des de threads)
# ==================
def task_move_to(expected_target: int):
"""
Segueix la línia fins arribar a expected_target (CROSSING o OBJECT).
Equivalent a task_move_to() del FreeRTOS.
"""
global wheels_speed_mode
speed = 0.0
wheels_speed_mode = True
enable_wheels(ON)
lf_response = CLEAR
while True:
speed = compute_new_speed(speed)
lf_response = follow_line_loop(speed)
if lf_response != CLEAR:
break
time.sleep(LINE_FOLLOWER_PERIOD)
if lf_response == CROSSING and expected_target == CROSSING:
run_to_crossing_center(speed)
elif lf_response == OBJECT and expected_target == OBJECT:
wheel_L.stop()
wheel_R.stop()
wheels_speed_mode = False
enable_wheels(OFF)
def task_rotate(direction: bool):
"""
Gira el robot 90° (CW o CCW).
Fase 1: acceleració fins al 90% dels passos de rotació.
Fase 2: desacceleració i ajust fi sobre la línia via sensors analògics.
"""
global wheels_speed_mode
# Arc 90° a Ø250mm → 2·π·125/4 ≈ 196mm; roda Ø152mm → 196/477 ≈ 0.41 voltes
ROTATION_STEPS = (WHEEL_STEPS_PER_REVOLUTION * 42) // 100
positive_wheel = wheel_L if direction == CW else wheel_R
speed = 0.0
wheels_speed_mode = True
enable_wheels(ON)
wheels_set_position(0)
while positive_wheel.current_position() < (ROTATION_STEPS * 90) // 100:
speed = compute_new_speed(speed)
wheels_set_speed(speed, rotate=True, direction=direction)
time.sleep(0.01)
speed /= 2
while True:
lf_r, lf_l = _read_line_sensors()
error = abs(lf_l - lf_r)
if error < ROTATION_ERROR_THRESHOLD or \
positive_wheel.current_position() > (ROTATION_STEPS * 110) // 100:
break
wheels_set_speed(speed - (speed / error), rotate=True, direction=direction)
time.sleep(0.01)
wheels_set_speed(0)
enable_wheels(OFF)
wheels_speed_mode = False
def task_take_or_leave_something(take: bool):
"""
Avança fins a l'objecte, baixa braços, opera la xeringa i torna al creuament.
take=True → xucla; take=False → buida.
"""
global wheels_speed_mode
speed = 0.0
wheels_speed_mode = True
enable_wheels(ON)
wheels_set_position(0) # guardem la posició home per tornar-hi
lf_response = CLEAR
while True:
speed = compute_new_speed(speed)
lf_response = follow_line_loop(speed)
if lf_response != CLEAR:
break
time.sleep(LINE_FOLLOWER_PERIOD)
if lf_response == OBJECT:
target_l = wheel_L.current_position() + mm_to_steps(MM_TO_OBJECT)
target_r = wheel_R.current_position() + mm_to_steps(MM_TO_OBJECT)
while wheel_L.current_position() < target_l and wheel_R.current_position() < target_r:
wheels_set_speed(speed / 2)
time.sleep(0.1)
wheels_set_speed(0)
enable_wheels(OFF)
enable_arms(ON)
move_arms_to(ARM_LOWER_POSITION)
enable_syringe(ON)
syringe.move_to(_SY_FULL_EXTENDED_STEPS if take else 0)
while syringe.distance_to_go() != 0:
if not take and is_endstop_detecting(END_SY):
syringe.stop()
syringe.set_current_position(0)
break
time.sleep(0.1)
enable_syringe(OFF)
move_arms_up()
enable_wheels(ON)
wheels_speed_mode = False
wheels_set_speed(WHEELS_MAX_SPEED)
move_wheels_to(0)
elif lf_response == CROSSING:
run_to_crossing_center(speed)
wheels_speed_mode = False
enable_wheels(OFF)
def task_idle():
"""Pausa breu fins que quibot.py assigni una nova tasca."""
time.sleep(0.5)

84
raspi/pins.py Normal file
View File

@@ -0,0 +1,84 @@
"""
pins.py — Definició de pins GPIO (BCM) de la Raspberry Pi Zero 2W.
Equivalent a io.h del codi Arduino/ESP32.
Tots els números fan referència a la numeració BCM.
"""
# ==================
# MOTORS
# ==================
# Servo d'expulsió de blocs
EN_SERVO = 9
SERVO_PWM = 10
# Motor pas a pas xeringa
STEP_SY = 8
DIR_SY = 5
# Motor pas a pas roda dreta
STEP_R_W = 25
DIR_R_W = 23
# Motor pas a pas roda esquerra
STEP_L_W = 7
DIR_L_W = 11
# Enable motors de rodes (compartit)
EN_W = 6
# Motor pas a pas braç dret
STEP_R_A = 3
DIR_R_A = 4
# Motor pas a pas braç esquerre
STEP_L_A = 13
DIR_L_A = 0
# Enable motors de braços (compartit)
EN_A = 21
# ==================
# SENSORS
# ==================
# Bus I2C principal — VL53L0X (distància) i PAJ7620U2 (gestos), compartit
SDA_DIST = 2
SCL_DIST = 1
SDA_GEST = SDA_DIST # mateixa línia
SCL_GEST = SCL_DIST # mateixa línia
# INT_GEST no connectat a la PCB — el driver usa polling
# Bus I2C sensor de color TCS34725 (bit-bang)
SDA_COL = 22
SCL_COL = 27
# Final de carrera xeringa (efecte Hall)
END_SY = 12
# Final de carrera braç dret (efecte Hall)
END_RA = 16
# Final de carrera braç esquerre (efecte Hall)
END_LA = 17
# Sensors seguidors de línia (TCRT5000)
LINES_R = 14
LINES_L = 15
# ==================
# DISPLAY
# ==================
# Dades matriu LED 8x8 RGB WS2811 (2x ull)
LED_DATA = 26
# ==================
# ÀUDIO (afegit per company, no usat pel robot)
# ==================
# I2S — amplificador MAX98357A + micròfon SPH0645
I2C_BCLK = 18
I2C_LRCLK = 19
AMP_DIN = 24
MIC = 20

286
raspi/quibot.py Normal file
View File

@@ -0,0 +1,286 @@
"""
quibot.py — Programa principal del robot QuiBot H2O.
Inicialitza tots els mòduls, executa el homing i arrenca els threads.
Equivalent a QuiBot.ino del codi Arduino/ESP32.
Threads permanents:
- task_read_blocks → llegeix blocs i executa accions (aquest fitxer)
- task_read_gestures → llegeix gestos i executa accions (aquest fitxer)
- _stepper_loop → genera polsos STEP dels motors (motion.py)
- _task_update_leds → parpelleig dels LEDs (eyes.py)
- _poll_loop → polling del sensor de gestos (gesture.py)
"""
import time
import threading
import signal
import sys
import pigpio
from motion import (
motion_setup, motion_cleanup,
arms_home, syringe_home,
task_move_to, task_rotate, task_take_or_leave_something, task_idle,
distance_to_object,
CROSSING, TAKE, LEAVE, CW, CCW,
)
from blocks import (
blocks_setup,
read_block_color, servo_move_to,
OPEN_POSITION, EJECT_POSITION,
BK, RD, GN, BU, YE, OG, VT,
)
from eyes import (
eyes_setup, eyes_cleanup,
eyes_turn_on, eyes_turn_off,
eyes_gesture_mode_on, eyes_gesture_mode_off,
EYES_OPEN, EYES_FW, EYES_DOWN,
RED, GREEN, BLUE, YELLOW, ORANGE, CYAN,
)
from gesture import (
gesture_setup, gesture_cleanup,
read_gesture,
GS_NONE, GS_FORWARD, GS_LEFT, GS_RIGHT,
GS_UP, GS_DOWN, GS_CLOCKWISE, GS_ANTICLOCKWISE, GS_WAVE,
)
# ==================
# Colors addicionals (equivalents CRGB del C++)
# ==================
GRAY = (128, 128, 128) # CRGB::Gray → estat normal
DARK_RED = (139, 0, 0) # CRGB::DarkRed → avançar
# ==================
# Timeouts
# ==================
INSERT_BLOCK_MS = 2.0 # s — espera que l'infant insereixi el bloc
EJECT_BLOCK_MS = 2.0 # s — espera que el bloc caigui
CHECK_COLOR_MS = 1.0 # s — interval entre lectures de color
# ==================
# Estat global
# ==================
_pi: pigpio.pi = None
# Mutex que evita que tasca de blocs i tasca de gestos llancin accions simultànies.
# En el C++ original compartien TaskHandle sense mutex explícit (possible race condition).
# Aquí ho fem correctament.
_action_lock = threading.Lock()
_shutdown_event = threading.Event()
_gesture_mode_active = False # False = mode blocs, True = mode gestos
_mode_lock = threading.Lock()
# ==================
# Helper d'execució d'accions
# ==================
def _execute_action(fn, *args):
"""
Adquireix el lock d'acció, executa fn(*args) de forma bloquejant i l'allibera.
Garanteix que mai s'executen dues accions de moviment simultànies.
"""
with _action_lock:
fn(*args)
# ==================
# Tasca de blocs
# ==================
def task_read_blocks():
"""
Llegeix blocs contínuament, executa l'acció corresponent al color i expulsa el bloc.
Equivalent a task_read_blocks() del FreeRTOS.
"""
while not _shutdown_event.is_set():
eyes_state = False
eyes_turn_on(EYES_OPEN, GRAY)
# Obre el servo per permetre la inserció del bloc
servo_move_to(OPEN_POSITION)
time.sleep(INSERT_BLOCK_MS)
# Espera que hi hagi un bloc i llegeix el seu color
color_id = BK
while not _shutdown_event.is_set():
if distance_to_object() < 80:
# Objecte a menys de 80mm — esperem que es retiri
if not eyes_state:
eyes_turn_on(EYES_DOWN, GRAY)
eyes_state = True
else:
color_id = read_block_color()
if eyes_state:
eyes_turn_on(EYES_OPEN, GRAY, 1, False)
eyes_state = False
if color_id != BK:
break
time.sleep(CHECK_COLOR_MS)
# Si estem en mode gestos, ignora el bloc
with _mode_lock:
if _gesture_mode_active:
continue
# Executa l'acció corresponent al color del bloc
if color_id == RD:
eyes_turn_on(EYES_FW, DARK_RED, 2)
time.sleep(1.0)
_execute_action(task_move_to, CROSSING)
elif color_id == GN:
eyes_turn_on(EYES_OPEN, GREEN, 2)
time.sleep(1.0)
_execute_action(task_rotate, CW)
elif color_id == BU:
eyes_turn_on(EYES_OPEN, BLUE, 2)
time.sleep(1.0)
_execute_action(task_rotate, CCW)
elif color_id == YE:
eyes_turn_on(EYES_OPEN, YELLOW, 2)
time.sleep(1.0)
_execute_action(task_take_or_leave_something, TAKE)
elif color_id == OG:
eyes_turn_on(EYES_OPEN, ORANGE, 2)
time.sleep(1.0)
_execute_action(task_take_or_leave_something, LEAVE)
elif color_id == VT:
_execute_action(task_idle)
eyes_turn_on(EYES_OPEN, GRAY)
# Expulsa el bloc
servo_move_to(EJECT_POSITION)
time.sleep(EJECT_BLOCK_MS)
# ==================
# Tasca de gestos
# ==================
def task_read_gestures():
"""
Llegeix gestos contínuament.
GS_WAVE activa/desactiva el mode gestos.
Equivalent a task_read_gestures() del FreeRTOS.
"""
gesture_mode_active = False
while not _shutdown_event.is_set():
gesture_id = read_gesture()
# WAVE: toggle entre mode blocs i mode gestos
if gesture_id == GS_WAVE:
with _mode_lock:
_gesture_mode_active = not _gesture_mode_active
active = _gesture_mode_active
if active:
eyes_gesture_mode_on()
print("Gesture mode ON")
else:
eyes_gesture_mode_off()
print("Gesture mode OFF")
time.sleep(1.0)
continue
with _mode_lock:
active = _gesture_mode_active
if not active or gesture_id == GS_NONE:
time.sleep(0.1)
continue
# Executa l'acció corresponent al gest
if gesture_id == GS_FORWARD:
eyes_turn_on(EYES_FW, DARK_RED, 2)
_execute_action(task_move_to, CROSSING)
elif gesture_id == GS_RIGHT:
eyes_turn_on(EYES_OPEN, GREEN, 2)
_execute_action(task_rotate, CW)
elif gesture_id == GS_LEFT:
eyes_turn_on(EYES_OPEN, BLUE, 2)
_execute_action(task_rotate, CCW)
elif gesture_id == GS_UP:
eyes_turn_on(EYES_OPEN, YELLOW, 2)
_execute_action(task_take_or_leave_something, TAKE)
elif gesture_id == GS_DOWN:
eyes_turn_on(EYES_OPEN, ORANGE, 2)
_execute_action(task_take_or_leave_something, LEAVE)
elif gesture_id in (GS_CLOCKWISE, GS_ANTICLOCKWISE):
_execute_action(task_idle)
eyes_turn_on(EYES_OPEN, CYAN)
time.sleep(0.5)
# ==================
# Shutdown
# ==================
def _shutdown(sig, frame):
print("\nAturant QuiBot...")
_shutdown_event.set()
# ==================
# Main
# ==================
def main():
global _pi
# Connecta amb pigpiod (ha d'estar en marxa amb: sudo pigpiod -s 1)
_pi = pigpio.pi()
if not _pi.connected:
print("ERROR: No s'ha pogut connectar a pigpiod. Executa: sudo pigpiod -s 1")
sys.exit(1)
# Inicialitza tots els mòduls
blocks_setup(_pi)
motion_setup(_pi)
eyes_setup(_pi)
gesture_setup()
# Homing (bloquejant)
arms_home()
syringe_home()
# Registra els senyals de sortida
signal.signal(signal.SIGINT, _shutdown)
signal.signal(signal.SIGTERM, _shutdown)
# Arrenca els threads principals
t_blocks = threading.Thread(target=task_read_blocks, daemon=True, name="blocks")
t_gestures = threading.Thread(target=task_read_gestures, daemon=True, name="gestures")
t_blocks.start()
t_gestures.start()
print("QuiBot llest.")
# Espera senyal de sortida
_shutdown_event.wait()
# Cleanup ordenat
motion_cleanup()
eyes_cleanup()
gesture_cleanup()
_pi.stop()
print("QuiBot aturat.")
if __name__ == "__main__":
main()

147
raspi/tests/test_blocks.py Normal file
View File

@@ -0,0 +1,147 @@
"""
test_blocks.py — Tests individuals del mòdul blocks.py.
Executa des del directori Rasp/: python tests/test_blocks.py
Descomenta la funció que vols provar al final del fitxer.
Assegura't que el venv està activat i pigpiod en marxa (sudo pigpiod -s 1).
"""
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
import time
import pigpio
from blocks import (
blocks_setup,
servo_move_to, OPEN_POSITION, EJECT_POSITION, MIN_SERVO_US, MAX_SERVO_US,
read_block_color, read_color_raw,
_COLORS,
BK, RD, GN, BU, YE, OG, VT,
)
_COLOR_NAMES = {
BK: "Negre (BK)",
RD: "Vermell (RD)",
GN: "Verd (GN)",
BU: "Blau (BU)",
YE: "Groc (YE)",
OG: "Taronja (OG)",
VT: "Violeta (VT)",
}
def _setup():
pi = pigpio.pi()
if not pi.connected:
print("ERROR: pigpiod no està en marxa. Executa: sudo pigpiod -s 1")
sys.exit(1)
blocks_setup(pi)
return pi
def _teardown(pi):
pi.stop()
# ==================
# TEST 1 — Servo
# ==================
def test_servo():
"""
Mou el servo a les posicions principals: oberta, expulsió i torna a oberta.
Hauries de veure/sentir el servo moure's suaument entre posicions.
"""
print("=== TEST SERVO ===")
pi = _setup()
print(f"Movent a OPEN_POSITION ({OPEN_POSITION} µs)...")
servo_move_to(OPEN_POSITION)
time.sleep(1.0)
print("Open: OK")
print(f"Movent a EJECT_POSITION ({EJECT_POSITION} µs)...")
servo_move_to(EJECT_POSITION)
time.sleep(1.0)
print("Eject: OK")
print(f"Tornant a OPEN_POSITION...")
servo_move_to(OPEN_POSITION)
time.sleep(1.0)
print("Torna a open: OK")
print(f"Provant posició mínima ({MIN_SERVO_US} µs)...")
servo_move_to(MIN_SERVO_US)
time.sleep(1.0)
print(f"Tornant a OPEN_POSITION...")
servo_move_to(OPEN_POSITION)
time.sleep(1.0)
_teardown(pi)
print("Test servo completat.\n")
# ==================
# TEST 2 — Sensor de color TCS34725
# ==================
def test_color_sensor():
"""
Llegeix el color 15 vegades cada segon.
Posa davant del sensor els blocs de colors per verificar que els reconeix.
"""
print("=== TEST SENSOR DE COLOR TCS34725 ===")
pi = _setup()
print("Llegint color durant 15 segons (posa els blocs davant del sensor)...")
for i in range(15):
color_id = read_block_color()
name = _COLOR_NAMES.get(color_id, "Desconegut")
print(f" Lectura {i+1:2d}: {name}")
time.sleep(1.0)
_teardown(pi)
print("Test sensor de color completat.\n")
# ==================
# TEST 3 — Calibració del sensor de color (valors RGB crus)
# ==================
def test_color_raw():
"""
Mostra valors RGB crus i la classificació actual durant 30 segons.
Útil per calibrar la taula _COLORS a blocks.py.
Format: R=xxx G=xxx B=xxx → classificat com 'XX' (diff=xx)
Si el diff és gran (>10), els valors de referència necessiten ajust.
"""
print("=== TEST COLOR RAW (calibració) ===")
pi = _setup()
print("Llegint valors RGB crus durant 30 segons (posa cada bloc davant del sensor)...")
print(f" {'R':>5} {'G':>5} {'B':>5} classificat diff")
for i in range(30):
r, g, b = read_color_raw()
best_name = "??"
best_diff = 9999
for ref in _COLORS:
diff = abs(r - ref["r"]) + abs(g - ref["g"]) + abs(b - ref["b"])
if diff < best_diff:
best_diff = diff
best_name = ref["name"]
print(f" R={r:3d} G={g:3d} B={b:3d} → {best_name} (diff={best_diff})")
time.sleep(1.0)
_teardown(pi)
print("Test color raw completat.\n")
# ==================
# Execució
# ==================
if __name__ == "__main__":
# Descomenta el test que vols executar:
test_servo()
# test_color_sensor()
# test_color_raw() # Per calibrar la taula de colors

156
raspi/tests/test_eyes.py Normal file
View File

@@ -0,0 +1,156 @@
"""
test_eyes.py — Tests individuals del mòdul eyes.py.
Executa des del directori Rasp/: python tests/test_eyes.py
Descomenta la funció que vols provar al final del fitxer.
Assegura't que el venv està activat i pigpiod en marxa (sudo pigpiod -s 1).
"""
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
import time
import pigpio
from eyes import (
eyes_setup, eyes_cleanup,
eyes_turn_on, eyes_turn_off,
eyes_gesture_mode_on, eyes_gesture_mode_off, eyes_listening,
EYES_OPEN, EYES_FW, EYES_DOWN, EYES_GESTURE,
WHITE, RED, GREEN, BLUE, YELLOW, ORANGE, PURPLE, CYAN, BLACK,
)
def _setup():
pi = pigpio.pi()
if not pi.connected:
print("ERROR: pigpiod no està en marxa. Executa: sudo pigpiod -s 1")
sys.exit(1)
eyes_setup(pi)
return pi
def _teardown(pi):
eyes_cleanup()
pi.stop()
# ==================
# TEST 1 — Formes i colors bàsics
# ==================
def test_shapes():
"""
Mostra totes les formes existents en colors diferents.
Hauries de veure les formes als ulls LED del robot.
"""
print("=== TEST FORMES I COLORS ===")
pi = _setup()
print("EYES_OPEN en blanc...")
eyes_turn_on(EYES_OPEN, WHITE)
time.sleep(2.0)
print("EYES_FW en vermell...")
eyes_turn_on(EYES_FW, RED)
time.sleep(2.0)
print("EYES_DOWN en blau...")
eyes_turn_on(EYES_DOWN, BLUE)
time.sleep(2.0)
print("EYES_OPEN en verd...")
eyes_turn_on(EYES_OPEN, GREEN)
time.sleep(2.0)
print("Apagant...")
eyes_turn_off()
time.sleep(1.0)
_teardown(pi)
print("Test formes completat.\n")
# ==================
# TEST 2 — Animació de repeat i direcció
# ==================
def test_animation():
"""
Prova l'animació amb repeat i les dues direccions.
Hauries de veure els LEDs encenent-se un per un en ordre normal i invers.
"""
print("=== TEST ANIMACIÓ ===")
pi = _setup()
print("EYES_OPEN groc, repeat=2, endavant...")
eyes_turn_on(EYES_OPEN, YELLOW, repeat=2, forward=True)
time.sleep(1.0)
print("EYES_FW taronja, repeat=2, enrere...")
eyes_turn_on(EYES_FW, ORANGE, repeat=2, forward=False)
time.sleep(1.0)
eyes_turn_off()
_teardown(pi)
print("Test animació completat.\n")
# ==================
# TEST 3 — Animacions mode gestos (TFG)
# ==================
def test_gesture_animations():
"""
Prova les animacions específiques del mode gestos.
Hauries de veure: doble parpelleig cian, tornada a blanc, marc cian.
"""
print("=== TEST ANIMACIONS MODE GESTOS ===")
pi = _setup()
print("Activant mode gestos (doble parpelleig cian)...")
eyes_gesture_mode_on()
time.sleep(2.0)
print("Escoltant gest (marc cian)...")
eyes_listening()
time.sleep(2.0)
print("Desactivant mode gestos (torna a blanc)...")
eyes_gesture_mode_off()
time.sleep(2.0)
eyes_turn_off()
_teardown(pi)
print("Test animacions gestos completat.\n")
# ==================
# TEST 4 — Parpelleig (breathing)
# ==================
def test_breathing():
"""
Verifica que el thread de parpelleig funciona correctament.
Hauries de veure la brillantor dels LEDs pujant i baixant suaument.
"""
print("=== TEST PARPELLEIG (BREATHING) ===")
pi = _setup()
print("EYES_OPEN blanc — observa el parpelleig durant 10 segons...")
eyes_turn_on(EYES_OPEN, WHITE)
time.sleep(10.0)
eyes_turn_off()
_teardown(pi)
print("Test parpelleig completat.\n")
# ==================
# Execució
# ==================
if __name__ == "__main__":
# Descomenta el test que vols executar:
test_shapes()
# test_animation()
# test_gesture_animations()
# test_breathing()

94
raspi/tests/test_gesture.py Normal file
View File

@@ -0,0 +1,94 @@
"""
test_gesture.py — Tests del mòdul gesture.py (sensor PAJ7620U2).
Executa des del directori Rasp/: python tests/test_gesture.py
Descomenta la funció que vols provar al final del fitxer.
Assegura't que el venv està activat i pigpiod en marxa (sudo pigpiod -s 1).
"""
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
import time
from gesture import (
gesture_setup, gesture_cleanup,
read_gesture,
GS_NONE, GS_FORWARD, GS_LEFT, GS_RIGHT,
GS_UP, GS_DOWN, GS_CLOCKWISE, GS_ANTICLOCKWISE, GS_WAVE,
)
_GESTURE_NAMES = {
GS_NONE: "Cap (GS_NONE)",
GS_FORWARD: "Endavant (GS_FORWARD)",
GS_LEFT: "Esquerra (GS_LEFT)",
GS_RIGHT: "Dreta (GS_RIGHT)",
GS_UP: "Amunt (GS_UP)",
GS_DOWN: "Avall (GS_DOWN)",
GS_CLOCKWISE: "Horari (GS_CLOCKWISE)",
GS_ANTICLOCKWISE: "Antihorari (GS_ANTICLOCKWISE)",
GS_WAVE: "Wave (GS_WAVE)",
}
# ==================
# TEST 1 — Connexió i inicialització
# ==================
def test_connection():
"""
Comprova que el sensor PAJ7620U2 és accessible via I2C.
Ha de mostrar 'Gesture sensor init OK' sense errors.
"""
print("=== TEST CONNEXIÓ PAJ7620U2 ===")
gesture_setup()
time.sleep(0.5)
gesture_cleanup()
print("Test connexió completat.\n")
# ==================
# TEST 2 — Lectura de gestos
# ==================
def test_read_gestures():
"""
Llegeix gestos durant 30 segons i els mostra per pantalla.
Fes gestos davant del sensor per verificar que els detecta correctament:
- Mà cap endavant/enrere → GS_FORWARD
- Mà cap a l'esquerra → GS_LEFT
- Mà cap a la dreta → GS_RIGHT
- Mà cap amunt → GS_UP
- Mà cap avall → GS_DOWN
- Rotació horària → GS_CLOCKWISE
- Rotació antihorària → GS_ANTICLOCKWISE
- Sacsejada (wave) → GS_WAVE
"""
print("=== TEST LECTURA GESTOS PAJ7620U2 ===")
print("Fes gestos davant del sensor durant 30 segons...")
gesture_setup()
time.sleep(0.5)
inici = time.time()
while time.time() - inici < 30:
gest = read_gesture()
if gest != GS_NONE:
nom = _GESTURE_NAMES.get(gest, f"Desconegut ({gest})")
print(f" Gest detectat: {nom}")
time.sleep(0.05)
gesture_cleanup()
print("Test lectura gestos completat.\n")
# ==================
# Execució
# ==================
if __name__ == "__main__":
# Descomenta el test que vols executar:
test_connection()
# test_read_gestures()

238
raspi/tests/test_motion.py Normal file
View File

@@ -0,0 +1,238 @@
"""
test_motion.py — Tests individuals del mòdul motion.py.
Executa des del directori Rasp/: python tests/test_motion.py
Descomenta la funció que vols provar al final del fitxer.
Assegura't que el venv està activat i pigpiod en marxa (sudo pigpiod -s 1).
"""
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
import time
import pigpio
import motion
from motion import (
motion_setup, motion_setup_steppers, motion_setup_sensors, motion_cleanup,
enable_wheels, enable_arms, enable_syringe,
arms_home, syringe_home,
distance_to_object,
ON, OFF, CW, CCW,
)
def _pi_connect():
pi = pigpio.pi()
if not pi.connected:
print("ERROR: pigpiod no està en marxa. Executa: sudo pigpiod -s 1")
sys.exit(1)
return pi
def _setup_motors():
"""Setup mínim per a tests de motors (sense sensors I2C)."""
pi = _pi_connect()
motion_setup_steppers(pi)
return pi
def _setup_sensors():
"""Setup mínim per a tests de sensors I2C (sense steppers)."""
pi = _pi_connect()
motion_setup_sensors(pi)
return pi
def _teardown_motors(pi):
motion_cleanup()
pi.stop()
def _teardown_sensors(pi):
pi.stop()
# ==================
# TEST 1 — Motors pas a pas (rodes)
# ==================
def test_motor_unic():
"""
Prova únicament la roda DRETA: 200 passos endavant i enrere.
Primer test a fer — verifica que un sol motor funciona correctament.
Hauries de veure la roda girar ~1/5 de volta (reducció 1:5).
"""
print("=== TEST MOTOR ÚNIC (RODA DRETA) ===")
pi = _setup_motors()
PASSOS = 200
print(f"Movent roda DRETA {PASSOS} passos endavant...")
enable_wheels(ON)
motion.wheel_R.move_to(PASSOS)
while motion.wheel_R.is_running():
time.sleep(0.05)
print("Roda dreta endavant: OK")
time.sleep(0.5)
print(f"Movent roda DRETA {PASSOS} passos enrere...")
motion.wheel_R.move_to(0)
while motion.wheel_R.is_running():
time.sleep(0.05)
print("Roda dreta enrere: OK")
enable_wheels(OFF)
_teardown_motors(pi)
print("Test motor únic completat.\n")
def test_motors():
"""
Prova les dues rodes: avança 200 passos i torna enrere.
Executa test_motor_unic() primer per verificar que un motor funciona.
Hauries de veure cada roda girar ~1/5 de volta (reducció 1:5).
"""
print("=== TEST MOTORS (RODES) ===")
pi = _setup_motors()
PASSOS = 200
print(f"Movent roda DRETA {PASSOS} passos endavant...")
enable_wheels(ON)
motion.wheel_R.move_to(PASSOS)
while motion.wheel_R.is_running():
time.sleep(0.05)
print("Roda dreta: OK")
time.sleep(0.5)
print(f"Movent roda DRETA {PASSOS} passos enrere...")
motion.wheel_R.move_to(0)
while motion.wheel_R.is_running():
time.sleep(0.05)
print("Roda dreta enrere: OK")
time.sleep(0.5)
print(f"Movent roda ESQUERRA {PASSOS} passos endavant...")
motion.wheel_L.move_to(PASSOS)
while motion.wheel_L.is_running():
time.sleep(0.05)
print("Roda esquerra: OK")
time.sleep(0.5)
print(f"Movent roda ESQUERRA {PASSOS} passos enrere...")
motion.wheel_L.move_to(0)
while motion.wheel_L.is_running():
time.sleep(0.05)
print("Roda esquerra enrere: OK")
enable_wheels(OFF)
_teardown_motors(pi)
print("Test motors completat.\n")
# ==================
# TEST 2 — Homing (finals de carrera)
# ==================
def test_homing_brac():
"""
Executa el homing únicament dels BRAÇOS.
Primer test de homing — verifica que un sol conjunt de finals de carrera funciona.
ATENCIÓ: assegura't que els braços tinguin espai per moure's.
"""
print("=== TEST HOMING BRAÇOS ===")
pi = _setup_motors()
print("Iniciant homing dels BRAÇOS...")
arms_home()
print("Homing braços: OK")
_teardown_motors(pi)
print("Test homing braços completat.\n")
def test_homing():
"""
Executa el homing complet: braços i xeringa.
Executa test_homing_brac() primer per verificar els finals de carrera dels braços.
ATENCIÓ: assegura't que els braços i la xeringa tinguin espai per moure's.
"""
print("=== TEST HOMING COMPLET ===")
pi = _setup_motors()
print("Iniciant homing dels BRAÇOS...")
arms_home()
print("Homing braços: OK")
time.sleep(1.0)
print("Iniciant homing de la XERINGA...")
syringe_home()
print("Homing xeringa: OK")
_teardown_motors(pi)
print("Test homing completat.\n")
# ==================
# TEST 3 — Sensor de distància VL53L0X
# ==================
def test_distance_sensor():
"""
Llegeix la distància 10 vegades cada 500ms.
Posa la mà davant del sensor per verificar que canvia el valor.
"""
print("=== TEST SENSOR DISTÀNCIA VL53L0X ===")
pi = _setup_sensors()
print("Llegint distància durant 5 segons (posa la mà davant del sensor)...")
for i in range(10):
dist = distance_to_object()
if dist == 65535:
print(f" Lectura {i+1:2d}: fora de rang")
else:
print(f" Lectura {i+1:2d}: {dist} mm")
time.sleep(0.5)
_teardown_sensors(pi)
print("Test sensor distància completat.\n")
# ==================
# TEST 4 — Sensors de línia ADS1115
# ==================
def test_line_sensors():
"""
Llegeix els dos sensors de línia 10 vegades cada 500ms.
Posa el sensor sobre superfícies de diferent color per veure la variació.
Valor alt (~9700+) = negre. Valor baix = blanc/clar.
"""
print("=== TEST SENSORS DE LÍNIA ADS1115 ===")
pi = _setup_sensors()
print("Llegint sensors de línia durant 5 segons...")
print(" (posa els sensors sobre blanc i negre per veure la diferència)")
for i in range(10):
r = motion._chan_r.value
l = motion._chan_l.value
print(f" Lectura {i+1:2d}: DRETA={r:5d} ESQUERRA={l:5d} error={r-l:+6d}")
time.sleep(0.5)
_teardown_sensors(pi)
print("Test sensors de línia completat.\n")
# ==================
# Execució
# ==================
if __name__ == "__main__":
# Descomenta el test que vols executar:
test_motor_unic() # Primer: prova un sol motor
# test_motors() # Segon: prova les dues rodes
# test_homing_brac() # Primer homing: només els braços
# test_homing() # Homing complet: braços i xeringa
# test_distance_sensor()
# test_line_sensors()

53
raspi/tests/test_simple_motor.py Normal file
View File

@@ -0,0 +1,53 @@
"""
test_simple_motor.py — Test mínim de la roda dreta per diagnosticar problemes.
Executa des del directori Rasp/tests/: python3 test_simple_motor.py
Diferències respecte a test_motion.py:
- Usa pi.write() directe en lloc de gpio_trigger()
- Bucle bloquejant amb time.sleep() en lloc de thread
- Sense acceleració, velocitat constant
"""
import pigpio
import time
STEP = 25 # STEP_R_W
DIR = 23 # DIR_R_W
EN = 6 # EN_W (actiu LOW)
pi = pigpio.pi()
if not pi.connected:
print("ERROR: pigpiod no està en marxa")
exit(1)
pi.set_mode(STEP, pigpio.OUTPUT)
pi.set_mode(DIR, pigpio.OUTPUT)
pi.set_mode(EN, pigpio.OUTPUT)
pi.write(EN, 0) # Activa el driver (LOW = ON)
pi.write(DIR, 1) # Endavant
# 500 passos/s → període = 1/500 = 2ms → mig període = 1ms
DELAY = 0.001
print("Movent 200 passos endavant...")
for _ in range(200):
pi.write(STEP, 1)
time.sleep(DELAY)
pi.write(STEP, 0)
time.sleep(DELAY)
print("Fet. Esperant 1s...")
time.sleep(1)
pi.write(DIR, 0) # Enrere
print("Movent 200 passos enrere...")
for _ in range(200):
pi.write(STEP, 1)
time.sleep(DELAY)
pi.write(STEP, 0)
time.sleep(DELAY)
print("Fet.")
pi.write(EN, 1) # Desactiva el driver
pi.stop()