Noxely se sépare en client et serveur (2.0.0)
Le téléphone balayait quarante radios toutes les trois minutes, cherchait dans quatre catalogues et enregistrait des directs. Tout cela le tenait éveillé pour un travail qui n'a aucune raison de se faire sur batterie. Un serveur reprend la collecte : recherche, guet, enregistrement, file de téléchargement, piloté depuis un navigateur. Le mobile demande ce qui a changé depuis son curseur, télécharge et se rendort — toutes les six heures, en Wi-Fi non facturé. Vingt fichiers quittent l'application au passage. Le balayage devient continu plutôt qu'espacé de trois minutes : une sonde ne lit que le premier bloc de métadonnées avant de refermer, ce qui rend la permanence tenable quand garder les flux ouverts coûterait plusieurs mégabits par seconde. Un morceau guetté a été capté en moins de vingt-cinq secondes. La recherche interroge les quatre catalogues à la fois et fond leurs résultats, les plus légers d'abord. Quatre défauts corrigés en chemin, tous trouvés en essayant plutôt qu'en relisant : un jeton régénéré à chaque démarrage, qui aurait obligé à reconfigurer le mobile ; un 403 de Wikimedia dû à un agent trop générique, que la fusion des sources masquait ; un dossier de données par défaut inaccessible au compte qui lance le serveur ; et une lecture de métadonnées qui n'itérait rien, faute d'être asynchrone.
This commit is contained in:
1 parent
adc6fc0486
commit
ed1d40826b
48 files changed
+2081
-3619
No files matched your search
@@ -0,0 +1,19 @@
|
||||
# Jeton partagé entre le serveur et l'application Android. Sans lui, l'API refuse tout.
|
||||
# Générer par exemple avec : openssl rand -base64 24
|
||||
NOXELY_TOKEN=
|
||||
|
||||
# Où sont rangés la base et les fichiers média.
|
||||
# Par défaut dans le projet, pour que `make server-run` fonctionne sans droits particuliers.
|
||||
# En service systemd, /var/lib/noxely est plus propre : l'unité le crée elle-même
|
||||
# (StateDirectory), inutile de le préparer à la main.
|
||||
NOXELY_DATA=./data
|
||||
|
||||
# Interface et port d'écoute.
|
||||
NOXELY_HOST=0.0.0.0
|
||||
NOXELY_PORT=8710
|
||||
|
||||
# Balayage des radios. Il est continu : un tour enchaîne sur le suivant.
|
||||
# NOXELY_WATCH_INTERVAL n'est qu'une pause de sécurité entre deux tours.
|
||||
NOXELY_WATCH_STATIONS=40
|
||||
NOXELY_WATCH_INTERVAL=5
|
||||
NOXELY_WATCH_CONCURRENCY=8
|
||||
@@ -0,0 +1,44 @@
|
||||
# 🖥️ Noxely — serveur
|
||||
|
||||
Le serveur porte tout ce qui coûtait de la batterie au téléphone : la recherche dans les
|
||||
catalogues, le guet des radios, l'enregistrement et le téléchargement. Le mobile ne fait plus que
|
||||
récupérer des fichiers prêts.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
make server-venv # crée le venv et installe les dépendances
|
||||
cp server/.env.example server/.env # puis renseigner NOXELY_TOKEN
|
||||
make server-run # lancement au premier plan, pour essayer
|
||||
```
|
||||
|
||||
L'interface de pilotage est à `http://<hôte>:8710` — recherche, guets, file et bibliothèque.
|
||||
Le jeton se saisit une fois dans l'en-tête de la page.
|
||||
|
||||
## En service
|
||||
|
||||
```bash
|
||||
make server-install # unité systemd (sudo demandé)
|
||||
make server-status # état
|
||||
make server-logs # journal suivi
|
||||
```
|
||||
|
||||
## API
|
||||
|
||||
| Route | Qui l'utilise |
|
||||
|---|---|
|
||||
| `GET /api/health` | tout le monde, sans jeton |
|
||||
| `GET /api/library?since=<curseur>` | **le mobile** — uniquement ce qui a changé |
|
||||
| `GET /api/media/{id}/file` | **le mobile** — le fichier |
|
||||
| `GET /api/search`, `/api/tracks`, `POST /api/queue` | l'interface web |
|
||||
| `GET/POST/DELETE /api/watches`, `POST /api/watches/sweep` | l'interface web |
|
||||
|
||||
Toutes les routes sauf `/api/health` exigent l'en-tête `X-Noxely-Token`.
|
||||
|
||||
## Données
|
||||
|
||||
```
|
||||
$NOXELY_DATA/
|
||||
├── noxely.sqlite3 index des médias et des guets
|
||||
└── media/ les fichiers eux-mêmes
|
||||
```
|
||||
@@ -0,0 +1,27 @@
|
||||
# Noxely — serveur : recherche, guet radio, enregistrement et mise à disposition hors ligne.
|
||||
# Installé par `make server-install` : les marqueurs @...@ sont remplacés à l'installation,
|
||||
# jamais figés dans le dépôt — le chemin et l'utilisateur diffèrent d'une machine à l'autre.
|
||||
|
||||
[Unit]
|
||||
Description=Noxely - serveur de collecte et de synchronisation
|
||||
Documentation=file://@PROJECT@/server/README.md
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=@USER@
|
||||
WorkingDirectory=@PROJECT@/server
|
||||
# NOXELY_TOKEN, NOXELY_DATA, NOXELY_HOST, NOXELY_PORT
|
||||
EnvironmentFile=@PROJECT@/server/.env
|
||||
Environment=PYTHONUNBUFFERED=1
|
||||
# Crée /var/lib/noxely avec les bons droits : rien à préparer à la main.
|
||||
StateDirectory=noxely
|
||||
ExecStart=@PROJECT@/server/venv/bin/python -m noxely_server
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
Whitespace-only changes.
@@ -0,0 +1,44 @@
|
||||
"""Point d'entrée : `python -m noxely_server`."""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
import uvicorn
|
||||
|
||||
from .config import load
|
||||
|
||||
|
||||
def ensure_usable(data_dir) -> None:
|
||||
"""Le dossier de données est la première chose qui peut manquer : on le dit clairement."""
|
||||
try:
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
probe = data_dir / ".ecriture"
|
||||
probe.touch()
|
||||
probe.unlink()
|
||||
except PermissionError:
|
||||
print(
|
||||
f"[noxely] Droits insuffisants sur {data_dir}.\n"
|
||||
f" Choisissez un dossier accessible dans server/.env (NOXELY_DATA=./data),\n"
|
||||
f" ou créez-le : sudo install -d -o $(whoami) {data_dir}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
raise SystemExit(1)
|
||||
except OSError as error:
|
||||
print(f"[noxely] Dossier de données inutilisable ({data_dir}) : {error}", file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
config = load()
|
||||
ensure_usable(config.data_dir)
|
||||
|
||||
# Importé après la vérification : l'application ouvre la base dès son chargement.
|
||||
from .api import app
|
||||
|
||||
print(f"[noxely] données : {config.data_dir}")
|
||||
print(f"[noxely] écoute : http://{config.host}:{config.port}")
|
||||
uvicorn.run(app, host=config.host, port=config.port, log_level="info")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,198 @@
|
||||
"""API HTTP du serveur Noxely.
|
||||
|
||||
Le client Android ne consomme que `/api/health`, `/api/library` et `/api/media/{id}/file` :
|
||||
tout le reste sert au pilotage depuis un navigateur. Cette séparation est volontaire — le
|
||||
mobile doit pouvoir se contenter d'un sondage bref, c'est là qu'est le gain de batterie.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import Body, Depends, FastAPI, Header, HTTPException, Query, Request
|
||||
from fastapi.responses import FileResponse, HTMLResponse
|
||||
|
||||
from . import config as config_module
|
||||
from .sources import catalogs, radio
|
||||
from .storage import Storage
|
||||
from .workers import Downloader, Watcher, safe_name
|
||||
|
||||
VERSION = "1.0.0"
|
||||
WEB_DIR = Path(__file__).parent / "web"
|
||||
|
||||
config = config_module.load()
|
||||
storage = Storage(config.db_path)
|
||||
downloader = Downloader(config, storage)
|
||||
watcher = Watcher(config, storage)
|
||||
|
||||
|
||||
def authorised(x_noxely_token: str = Header(default="")) -> None:
|
||||
"""Sans jeton valide, le serveur ne répond rien : il expose des fichiers."""
|
||||
if x_noxely_token != config.token:
|
||||
raise HTTPException(status_code=401, detail="Jeton invalide")
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
tasks = [
|
||||
asyncio.create_task(downloader.run_forever()),
|
||||
asyncio.create_task(watcher.run_forever()),
|
||||
]
|
||||
yield
|
||||
for task in tasks:
|
||||
task.cancel()
|
||||
|
||||
|
||||
app = FastAPI(title="Noxely", version=VERSION, lifespan=lifespan)
|
||||
|
||||
|
||||
# --- Ce que consomme le mobile ------------------------------------------
|
||||
|
||||
@app.get("/api/health")
|
||||
def health() -> dict:
|
||||
return {"name": "Noxely", "version": VERSION}
|
||||
|
||||
|
||||
@app.get("/api/library", dependencies=[Depends(authorised)])
|
||||
def library(since: int = Query(default=0)) -> dict:
|
||||
"""Uniquement ce qui a changé depuis le dernier passage du client."""
|
||||
items = storage.media_since(since)
|
||||
return {
|
||||
"items": [m.public() for m in items],
|
||||
"cursor": items[-1].updated_at if items else since,
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/media/{media_id}/file", dependencies=[Depends(authorised)])
|
||||
def media_file(media_id: str):
|
||||
item = storage.media(media_id)
|
||||
if not item or item.state != "ready":
|
||||
raise HTTPException(status_code=404, detail="Média indisponible")
|
||||
path = config.media_dir / item.filename
|
||||
if not path.exists():
|
||||
raise HTTPException(status_code=404, detail="Fichier absent")
|
||||
return FileResponse(path, media_type=item.mime, filename=item.filename)
|
||||
|
||||
|
||||
@app.delete("/api/media/{media_id}", dependencies=[Depends(authorised)])
|
||||
def delete_media(media_id: str) -> dict:
|
||||
item = storage.delete_media(media_id)
|
||||
if item:
|
||||
(config.media_dir / item.filename).unlink(missing_ok=True)
|
||||
return {"deleted": bool(item)}
|
||||
|
||||
|
||||
# --- Pilotage depuis le navigateur --------------------------------------
|
||||
|
||||
@app.get("/api/catalogs", dependencies=[Depends(authorised)])
|
||||
def list_catalogs() -> dict:
|
||||
return {
|
||||
"catalogs": {"all": "Toutes les sources", **catalogs.CATALOGS},
|
||||
"genres": radio.GENRES,
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/search", dependencies=[Depends(authorised)])
|
||||
async def search(catalog: str, q: str, language: str | None = None) -> dict:
|
||||
try:
|
||||
if catalog == "all":
|
||||
return {"results": await catalogs.search_all(q, language)}
|
||||
return {"results": await catalogs.search(catalog, q, language)}
|
||||
except Exception as error:
|
||||
raise HTTPException(status_code=502, detail=str(error)[:200])
|
||||
|
||||
|
||||
@app.get("/api/tracks", dependencies=[Depends(authorised)])
|
||||
async def tracks(catalog: str, id: str, title: str = "", kind: str = "Audio") -> dict:
|
||||
try:
|
||||
return {"tracks": await catalogs.tracks(catalog, id, title, kind)}
|
||||
except Exception as error:
|
||||
raise HTTPException(status_code=502, detail=str(error)[:200])
|
||||
|
||||
|
||||
@app.post("/api/queue", dependencies=[Depends(authorised)])
|
||||
def queue(payload: dict = Body(...)) -> dict:
|
||||
"""Met des pistes en file : le téléchargement se fait en tâche de fond."""
|
||||
added = 0
|
||||
for track in payload.get("tracks", []):
|
||||
url = track.get("url")
|
||||
if not url:
|
||||
continue
|
||||
extension = url.split("?")[0].rsplit(".", 1)[-1].lower()
|
||||
if extension not in ("mp3", "m4a", "ogg", "opus", "flac", "wav", "mp4", "webm"):
|
||||
extension = "mp3"
|
||||
storage.add_media(
|
||||
title=track.get("title") or "Sans titre",
|
||||
artist=track.get("artist") or "",
|
||||
source=payload.get("catalog") or "catalogue",
|
||||
origin=payload.get("origin") or "",
|
||||
filename=safe_name(track.get("title") or "media", extension),
|
||||
remote_url=url,
|
||||
duration_ms=int(track.get("duration_ms") or 0),
|
||||
artwork_url=track.get("artwork_url"),
|
||||
)
|
||||
added += 1
|
||||
return {"queued": added}
|
||||
|
||||
|
||||
@app.get("/api/radio/stations", dependencies=[Depends(authorised)])
|
||||
async def radio_stations(genre: str | None = None, name: str | None = None) -> dict:
|
||||
return {"stations": await radio.stations(genre, name)}
|
||||
|
||||
|
||||
@app.get("/api/watches", dependencies=[Depends(authorised)])
|
||||
def watches() -> dict:
|
||||
return {
|
||||
"watches": storage.watches(),
|
||||
"recording": watcher.recording,
|
||||
"sweeps": watcher.sweeps,
|
||||
"last_sweep": watcher.last_sweep,
|
||||
}
|
||||
|
||||
|
||||
@app.post("/api/watches", dependencies=[Depends(authorised)])
|
||||
def add_watch(payload: dict = Body(...)) -> dict:
|
||||
query = (payload.get("query") or "").strip()
|
||||
if not query:
|
||||
raise HTTPException(status_code=400, detail="Requête vide")
|
||||
return {"id": storage.add_watch(query)}
|
||||
|
||||
|
||||
@app.delete("/api/watches/{watch_id}", dependencies=[Depends(authorised)])
|
||||
def delete_watch(watch_id: str) -> dict:
|
||||
storage.delete_watch(watch_id)
|
||||
return {"deleted": True}
|
||||
|
||||
|
||||
@app.post("/api/watches/sweep", dependencies=[Depends(authorised)])
|
||||
async def sweep() -> dict:
|
||||
try:
|
||||
return await watcher.sweep()
|
||||
except Exception as error:
|
||||
# L'annuaire peut être momentanément injoignable : le dire, pas renvoyer une trace.
|
||||
raise HTTPException(status_code=502, detail=f"Annuaire indisponible : {str(error)[:120]}")
|
||||
|
||||
|
||||
@app.get("/api/queue", dependencies=[Depends(authorised)])
|
||||
def queue_state() -> dict:
|
||||
pending = storage.media_since(0, state=None)
|
||||
return {"items": [m.public() for m in pending if m.state != "ready"][:50]}
|
||||
|
||||
|
||||
@app.get("/api/bootstrap")
|
||||
def bootstrap(request: Request) -> dict:
|
||||
"""Livre le jeton — uniquement à une requête venue de la machine elle-même.
|
||||
|
||||
Depuis un autre poste, il faut le saisir : c'est précisément ce que le jeton protège.
|
||||
`make server-token` l'affiche pour le copier.
|
||||
"""
|
||||
client = request.client.host if request.client else ""
|
||||
if client not in ("127.0.0.1", "::1", "localhost"):
|
||||
raise HTTPException(status_code=403, detail="Jeton lisible seulement en local")
|
||||
return {"token": config.token}
|
||||
|
||||
|
||||
@app.get("/", response_class=HTMLResponse)
|
||||
def index() -> str:
|
||||
return (WEB_DIR / "index.html").read_text(encoding="utf-8")
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Configuration du serveur Noxely, lue de l'environnement.
|
||||
|
||||
Aucune valeur sensible n'est écrite dans le dépôt : le jeton et les chemins viennent d'un
|
||||
fichier .env local, à l'image des autres services de la maison.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import secrets
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _persisted_token(data_dir: Path) -> str:
|
||||
"""Jeton auto-généré, conservé entre deux démarrages.
|
||||
|
||||
Un jeton tiré à chaque lancement obligerait à reconfigurer le téléphone après le moindre
|
||||
redémarrage : il est donc écrit une fois, en lecture seule pour son propriétaire.
|
||||
"""
|
||||
path = data_dir / "token"
|
||||
if path.exists():
|
||||
existing = path.read_text().strip()
|
||||
if existing:
|
||||
return existing
|
||||
token = secrets.token_urlsafe(24)
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(token + "\n")
|
||||
path.chmod(0o600)
|
||||
print(f"[noxely] jeton généré et conservé dans {path}")
|
||||
return token
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Config:
|
||||
data_dir: Path
|
||||
token: str
|
||||
host: str
|
||||
port: int
|
||||
#: Nombre de stations balayées à chaque passage du guet.
|
||||
watch_stations: int
|
||||
#: Pause entre deux balayages, en secondes. Le balayage est continu : cette pause n'est
|
||||
#: qu'un garde-fou pour ne pas marteler l'annuaire si un tour échoue instantanément.
|
||||
watch_interval: int
|
||||
#: Stations sondées en parallèle. Plus il est haut, plus un tour complet est bref.
|
||||
watch_concurrency: int
|
||||
|
||||
@property
|
||||
def media_dir(self) -> Path:
|
||||
return self.data_dir / "media"
|
||||
|
||||
@property
|
||||
def db_path(self) -> Path:
|
||||
return self.data_dir / "noxely.sqlite3"
|
||||
|
||||
|
||||
def load() -> Config:
|
||||
data_dir = Path(os.environ.get("NOXELY_DATA", "./data")).expanduser().resolve()
|
||||
token = os.environ.get("NOXELY_TOKEN", "").strip() or _persisted_token(data_dir)
|
||||
return Config(
|
||||
data_dir=data_dir,
|
||||
token=token,
|
||||
host=os.environ.get("NOXELY_HOST", "0.0.0.0"),
|
||||
port=int(os.environ.get("NOXELY_PORT", "8710")),
|
||||
watch_stations=int(os.environ.get("NOXELY_WATCH_STATIONS", "40")),
|
||||
watch_interval=int(os.environ.get("NOXELY_WATCH_INTERVAL", "5")),
|
||||
watch_concurrency=int(os.environ.get("NOXELY_WATCH_CONCURRENCY", "8")),
|
||||
)
|
||||
@@ -0,0 +1,6 @@
|
||||
"""Catalogues libres et annuaire radio, portés depuis le client Android.
|
||||
|
||||
Les pièges vérifiés côté mobile restent valables ici et sont conservés en commentaire :
|
||||
LibriVox ignore son paramètre `language`, archive.org sert des éléments en accès restreint,
|
||||
et un titre ICY court jusqu'au « '; » et non jusqu'à la première apostrophe.
|
||||
"""
|
||||
@@ -0,0 +1,235 @@
|
||||
"""Recherche dans les catalogues libres de droits."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
import urllib.parse
|
||||
|
||||
import httpx
|
||||
|
||||
# Wikimedia refuse (403) les agents génériques : sa politique demande un nom identifiable
|
||||
# et un moyen de contact. Les autres services s'en accommodent aussi.
|
||||
UA = {"User-Agent": "Noxely/1.0 (https://git.tuxtech.fr/tuxgyver/Noxely)"}
|
||||
TIMEOUT = httpx.Timeout(30.0, connect=15.0)
|
||||
|
||||
AUDIO_EXT = (".mp3", ".m4a", ".ogg", ".opus", ".flac", ".wav")
|
||||
VIDEO_EXT = (".mp4", ".m4v", ".webm", ".ogv")
|
||||
|
||||
CATALOGS = {
|
||||
"archive": "Internet Archive",
|
||||
"underground": "Underground (netlabels)",
|
||||
"librivox": "LibriVox",
|
||||
"wikimedia": "Wikimedia Commons",
|
||||
}
|
||||
|
||||
|
||||
async def search_all(query: str, language: str | None = None) -> list[dict]:
|
||||
"""Interroge les quatre catalogues en parallèle et fond les résultats.
|
||||
|
||||
Chaque ligne porte son catalogue d'origine : c'est lui qu'il faudra pour en lister les
|
||||
pistes. Un catalogue en panne n'empêche pas les autres de répondre — on préfère une liste
|
||||
incomplète à une erreur globale.
|
||||
"""
|
||||
names = list(CATALOGS)
|
||||
outcomes = await asyncio.gather(
|
||||
*(search(name, query, language) for name in names), return_exceptions=True
|
||||
)
|
||||
merged: list[dict] = []
|
||||
for name, outcome in zip(names, outcomes):
|
||||
if isinstance(outcome, Exception):
|
||||
continue
|
||||
for item in outcome:
|
||||
merged.append({**item, "catalog": name})
|
||||
# Les résultats les plus légers d'abord : sur un téléphone, la taille prime.
|
||||
merged.sort(key=lambda item: item["size"] or 1 << 62)
|
||||
return merged
|
||||
|
||||
|
||||
async def search(catalog: str, query: str, language: str | None = None) -> list[dict]:
|
||||
if catalog == "librivox":
|
||||
return await _librivox(query, language)
|
||||
if catalog == "wikimedia":
|
||||
return await _wikimedia(query)
|
||||
collections = ["netlabels", "freemusicarchive"] if catalog == "underground" else []
|
||||
return await _archive(query, language, collections)
|
||||
|
||||
|
||||
async def tracks(catalog: str, item_id: str, title: str = "", kind: str = "Audio") -> list[dict]:
|
||||
"""Les fichiers d'un résultat : un album en a plusieurs, un morceau un seul."""
|
||||
if catalog == "librivox":
|
||||
return await _librivox_tracks(item_id)
|
||||
if catalog == "wikimedia":
|
||||
return [{"title": title or item_id, "artist": "Wikimedia Commons", "url": item_id, "size": 0}]
|
||||
return await _archive_tracks(item_id, title, kind)
|
||||
|
||||
|
||||
# --- archive.org --------------------------------------------------------
|
||||
|
||||
async def _archive(query: str, language: str | None, collections: list[str]) -> list[dict]:
|
||||
clauses = [f"({query})", "mediatype:(audio OR movies)"]
|
||||
if language:
|
||||
clauses.append(f"language:({language})")
|
||||
if collections:
|
||||
clauses.append("collection:(" + " OR ".join(collections) + ")")
|
||||
# Les éléments restreints répondent 401 au téléchargement : écartés dès la recherche.
|
||||
clauses.append("NOT access-restricted-item:true")
|
||||
|
||||
params = {
|
||||
"q": " AND ".join(clauses),
|
||||
"fl[]": ["identifier", "title", "creator", "mediatype", "item_size"],
|
||||
"rows": "30", "page": "1", "output": "json",
|
||||
}
|
||||
async with httpx.AsyncClient(headers=UA, timeout=TIMEOUT) as client:
|
||||
response = await client.get("https://archive.org/advancedsearch.php", params=params)
|
||||
response.raise_for_status()
|
||||
docs = response.json().get("response", {}).get("docs", [])
|
||||
|
||||
return [
|
||||
{
|
||||
"id": doc["identifier"],
|
||||
"title": _first(doc.get("title")) or doc["identifier"],
|
||||
"artist": _first(doc.get("creator")) or "Internet Archive",
|
||||
"kind": "Vidéo" if doc.get("mediatype") == "movies" else "Audio",
|
||||
"catalog": "archive",
|
||||
"size": int(doc.get("item_size") or 0),
|
||||
}
|
||||
for doc in docs if doc.get("identifier")
|
||||
]
|
||||
|
||||
|
||||
async def _archive_tracks(identifier: str, title: str, kind: str) -> list[dict]:
|
||||
async with httpx.AsyncClient(headers=UA, timeout=TIMEOUT, follow_redirects=True) as client:
|
||||
response = await client.get(f"https://archive.org/metadata/{identifier}")
|
||||
response.raise_for_status()
|
||||
meta = response.json()
|
||||
|
||||
if str(meta.get("metadata", {}).get("access-restricted-item", "")).lower() == "true":
|
||||
raise ValueError(f"« {title or identifier} » est en accès restreint")
|
||||
|
||||
wants_video = kind == "Vidéo"
|
||||
extensions = VIDEO_EXT if wants_video else AUDIO_EXT
|
||||
files = [f for f in meta.get("files", []) if f.get("name", "").lower().endswith(extensions)]
|
||||
if not files:
|
||||
raise ValueError("Aucun fichier lisible dans cet élément")
|
||||
|
||||
# Un élément expose l'original et ses dérivés : on garde le plus léger de chaque morceau,
|
||||
# autant de pistes que d'œuvres et non que de formats.
|
||||
groups: dict[str, dict] = {}
|
||||
for entry in files:
|
||||
stem = entry["name"].rsplit(".", 1)[0].lower()
|
||||
size = int(entry.get("size") or 1 << 62)
|
||||
if stem not in groups or size < int(groups[stem].get("size") or 1 << 62):
|
||||
groups[stem] = entry
|
||||
|
||||
out = []
|
||||
for entry in sorted(groups.values(), key=lambda f: f["name"]):
|
||||
name = entry["name"]
|
||||
out.append({
|
||||
"title": entry.get("title") or name.rsplit(".", 1)[0],
|
||||
"artist": title or identifier,
|
||||
"url": f"https://archive.org/download/{identifier}/" + urllib.parse.quote(name),
|
||||
"size": int(entry.get("size") or 0),
|
||||
"duration_ms": int(float(entry.get("length") or 0) * 1000),
|
||||
"artwork_url": f"https://archive.org/services/img/{identifier}",
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
# --- LibriVox -----------------------------------------------------------
|
||||
|
||||
async def _librivox(query: str, language: str | None) -> list[dict]:
|
||||
term = urllib.parse.quote(query.strip())
|
||||
words = query.strip().split()
|
||||
# `title` est une correspondance exacte sans « ^ », et `author` ne matche que le nom de
|
||||
# famille : trois interrogations valent mieux qu'une qui échouerait.
|
||||
criteria = [f"title=^{term}", f"author={term}"]
|
||||
if len(words) > 1:
|
||||
criteria.append("author=" + urllib.parse.quote(words[-1]))
|
||||
|
||||
seen: dict[str, dict] = {}
|
||||
limit = 25 if not language else 100
|
||||
async with httpx.AsyncClient(headers=UA, timeout=TIMEOUT) as client:
|
||||
for criterion in criteria:
|
||||
url = f"https://librivox.org/api/feed/audiobooks/?format=json&limit={limit}&{criterion}"
|
||||
response = await client.get(url)
|
||||
if response.status_code == 404: # « aucun résultat », pas une panne
|
||||
continue
|
||||
response.raise_for_status()
|
||||
for book in response.json().get("books", []):
|
||||
# Le paramètre `language` de l'API est accepté puis ignoré : filtrage ici.
|
||||
if language and (book.get("language") or "").lower() != language.lower():
|
||||
continue
|
||||
feed = book.get("url_rss")
|
||||
if feed and feed not in seen:
|
||||
authors = ", ".join(
|
||||
" ".join(filter(None, [a.get("first_name"), a.get("last_name")]))
|
||||
for a in book.get("authors", [])
|
||||
)
|
||||
seen[feed] = {
|
||||
"id": feed,
|
||||
"title": book.get("title") or "Livre audio",
|
||||
"artist": authors or "LibriVox",
|
||||
"kind": "Audio",
|
||||
"size": 0,
|
||||
"catalog": "librivox",
|
||||
}
|
||||
return list(seen.values())
|
||||
|
||||
|
||||
async def _librivox_tracks(feed_url: str) -> list[dict]:
|
||||
async with httpx.AsyncClient(headers=UA, timeout=TIMEOUT, follow_redirects=True) as client:
|
||||
response = await client.get(feed_url)
|
||||
response.raise_for_status()
|
||||
xml = response.text
|
||||
|
||||
title = re.search(r"<title>(.*?)</title>", xml, re.S)
|
||||
book = (title.group(1).strip() if title else "Livre audio")
|
||||
out = []
|
||||
for item in re.findall(r"<item>(.*?)</item>", xml, re.S):
|
||||
url = re.search(r'<enclosure[^>]*url="([^"]+)"', item)
|
||||
name = re.search(r"<title>(.*?)</title>", item, re.S)
|
||||
if url:
|
||||
out.append({
|
||||
"title": (name.group(1).strip() if name else "Épisode"),
|
||||
"artist": book,
|
||||
"url": url.group(1),
|
||||
"size": 0,
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
# --- Wikimedia Commons --------------------------------------------------
|
||||
|
||||
async def _wikimedia(query: str) -> list[dict]:
|
||||
params = {
|
||||
"action": "query", "format": "json", "generator": "search",
|
||||
"gsrsearch": f"filetype:audio {query.strip()}", "gsrnamespace": "6", "gsrlimit": "30",
|
||||
"prop": "imageinfo", "iiprop": "url|size|mime|extmetadata",
|
||||
}
|
||||
async with httpx.AsyncClient(headers=UA, timeout=TIMEOUT) as client:
|
||||
response = await client.get("https://commons.wikimedia.org/w/api.php", params=params)
|
||||
response.raise_for_status()
|
||||
pages = response.json().get("query", {}).get("pages", {})
|
||||
|
||||
out = []
|
||||
for page in pages.values():
|
||||
info = (page.get("imageinfo") or [{}])[0]
|
||||
url = info.get("url")
|
||||
if not url or not url.split("?")[0].lower().endswith(AUDIO_EXT):
|
||||
continue
|
||||
licence = (info.get("extmetadata", {}).get("LicenseShortName", {}) or {}).get("value")
|
||||
out.append({
|
||||
"id": url,
|
||||
"title": page.get("title", "").removeprefix("File:").rsplit(".", 1)[0],
|
||||
"artist": licence or "Wikimedia Commons",
|
||||
"kind": "Audio",
|
||||
"size": int(info.get("size") or 0),
|
||||
"catalog": "wikimedia",
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
def _first(value):
|
||||
if isinstance(value, list):
|
||||
return value[0] if value else None
|
||||
return value
|
||||
@@ -0,0 +1,92 @@
|
||||
"""Annuaire des stations et lecture des métadonnées ICY."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
import urllib.parse
|
||||
|
||||
import httpx
|
||||
|
||||
UA = {"User-Agent": "Noxely/1.0 (https://git.tuxtech.fr/tuxgyver/Noxely)"}
|
||||
BASE = "https://de1.api.radio-browser.info/json/stations/search"
|
||||
|
||||
#: Un titre ICY court jusqu'au « '; » : les apostrophes y sont trop fréquentes pour s'arrêter
|
||||
#: à la première (« Ain't That a Kick in the Head » deviendrait « Ain »).
|
||||
STREAM_TITLE = re.compile(r"StreamTitle='(.*?)';")
|
||||
|
||||
GENRES = ["pop", "rock", "jazz", "classical", "electronic", "news", "chanson"]
|
||||
|
||||
|
||||
async def stations(genre: str | None = None, name: str | None = None, limit: int = 60) -> list[dict]:
|
||||
params = {"limit": str(limit), "hidebroken": "true", "order": "clickcount", "reverse": "true"}
|
||||
if genre:
|
||||
params["tag"] = genre
|
||||
if name:
|
||||
params["name"] = name
|
||||
|
||||
# L'annuaire est un service communautaire : une coupure passagère ne doit pas faire
|
||||
# tomber le balayage. Mesuré en conditions réelles, d'où cette seconde tentative.
|
||||
rows = None
|
||||
async with httpx.AsyncClient(headers=UA, timeout=httpx.Timeout(30.0)) as client:
|
||||
for attempt in range(2):
|
||||
try:
|
||||
response = await client.get(BASE, params=params)
|
||||
response.raise_for_status()
|
||||
rows = response.json()
|
||||
break
|
||||
except Exception:
|
||||
if attempt == 1:
|
||||
raise
|
||||
await asyncio.sleep(2)
|
||||
|
||||
out = []
|
||||
for row in rows:
|
||||
url = row.get("url_resolved") or ""
|
||||
# Les flux HLS se lisent mais ne s'enregistrent pas en copiant les octets.
|
||||
if int(row.get("hls") or 0) != 0 or not url.startswith("http"):
|
||||
continue
|
||||
out.append({
|
||||
"id": row.get("stationuuid") or url,
|
||||
"name": (row.get("name") or "Station").strip(),
|
||||
"url": url,
|
||||
"codec": row.get("codec") or "",
|
||||
"bitrate": int(row.get("bitrate") or 0),
|
||||
"country": row.get("country") or "",
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
async def now_playing(stream_url: str) -> str | None:
|
||||
"""Le morceau diffusé à l'instant : aucun annuaire ne l'expose, il faut ouvrir le flux."""
|
||||
headers = {**UA, "Icy-MetaData": "1"}
|
||||
try:
|
||||
async with httpx.AsyncClient(headers=headers, timeout=httpx.Timeout(10.0)) as client:
|
||||
async with client.stream("GET", stream_url) as response:
|
||||
if response.status_code >= 300:
|
||||
return None
|
||||
interval = int(response.headers.get("icy-metaint") or 0)
|
||||
if interval <= 0:
|
||||
return None
|
||||
return await _read_title(response, interval)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
async def _read_title(response, interval: int) -> str | None:
|
||||
# Client asynchrone : `iter_bytes` n'itère rien ici, il faut `aiter_bytes`.
|
||||
buffer = bytearray()
|
||||
needed = interval + 1
|
||||
async for chunk in response.aiter_bytes():
|
||||
buffer.extend(chunk)
|
||||
if len(buffer) < needed:
|
||||
continue
|
||||
length = buffer[interval] * 16
|
||||
if length == 0:
|
||||
return None
|
||||
needed = interval + 1 + length
|
||||
if len(buffer) < needed:
|
||||
continue
|
||||
meta = bytes(buffer[interval + 1:needed]).decode("utf-8", "replace")
|
||||
found = STREAM_TITLE.search(meta)
|
||||
return found.group(1).strip() or None if found else None
|
||||
return None
|
||||
@@ -0,0 +1,179 @@
|
||||
"""Index des médias et des guets, en SQLite.
|
||||
|
||||
Les fichiers vivent sur le disque ; la base ne porte que leurs métadonnées et l'état de la
|
||||
synchronisation. Un client ne demande que ce qui a changé depuis son dernier passage, d'où le
|
||||
champ `updated_at` sur chaque ligne.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
import time
|
||||
import uuid
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
|
||||
SCHEMA = """
|
||||
CREATE TABLE IF NOT EXISTS media (
|
||||
id TEXT PRIMARY KEY,
|
||||
title TEXT NOT NULL,
|
||||
artist TEXT NOT NULL DEFAULT '',
|
||||
source TEXT NOT NULL,
|
||||
origin TEXT NOT NULL DEFAULT '',
|
||||
filename TEXT NOT NULL,
|
||||
mime TEXT NOT NULL DEFAULT 'audio/mpeg',
|
||||
size INTEGER NOT NULL DEFAULT 0,
|
||||
duration_ms INTEGER NOT NULL DEFAULT 0,
|
||||
artwork_url TEXT,
|
||||
state TEXT NOT NULL DEFAULT 'pending',
|
||||
error TEXT,
|
||||
remote_url TEXT,
|
||||
created_at INTEGER NOT NULL,
|
||||
updated_at INTEGER NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS media_updated ON media(updated_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS watches (
|
||||
id TEXT PRIMARY KEY,
|
||||
query TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
scans INTEGER NOT NULL DEFAULT 0,
|
||||
last_scan INTEGER,
|
||||
found_at INTEGER,
|
||||
found_on TEXT,
|
||||
found_title TEXT
|
||||
);
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class Media:
|
||||
id: str
|
||||
title: str
|
||||
artist: str
|
||||
source: str
|
||||
origin: str
|
||||
filename: str
|
||||
mime: str
|
||||
size: int
|
||||
duration_ms: int
|
||||
artwork_url: str | None
|
||||
state: str
|
||||
error: str | None
|
||||
remote_url: str | None
|
||||
created_at: int
|
||||
updated_at: int
|
||||
|
||||
def public(self) -> dict:
|
||||
"""Vue destinée au client : ni chemin local, ni URL d'origine."""
|
||||
data = asdict(self)
|
||||
data.pop("remote_url")
|
||||
data.pop("filename")
|
||||
return data
|
||||
|
||||
|
||||
def now() -> int:
|
||||
return int(time.time() * 1000)
|
||||
|
||||
|
||||
class Storage:
|
||||
def __init__(self, db_path: Path) -> None:
|
||||
db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._db = sqlite3.connect(db_path, check_same_thread=False)
|
||||
self._db.row_factory = sqlite3.Row
|
||||
self._db.executescript(SCHEMA)
|
||||
self._db.commit()
|
||||
|
||||
# --- Médias ---------------------------------------------------------
|
||||
|
||||
def add_media(self, *, title: str, artist: str, source: str, origin: str,
|
||||
filename: str, remote_url: str | None, mime: str = "audio/mpeg",
|
||||
duration_ms: int = 0, artwork_url: str | None = None,
|
||||
state: str = "pending") -> str:
|
||||
media_id = uuid.uuid4().hex
|
||||
stamp = now()
|
||||
self._db.execute(
|
||||
"INSERT INTO media (id,title,artist,source,origin,filename,mime,size,duration_ms,"
|
||||
"artwork_url,state,error,remote_url,created_at,updated_at) "
|
||||
"VALUES (?,?,?,?,?,?,?,0,?,?,?,NULL,?,?,?)",
|
||||
(media_id, title, artist, source, origin, filename, mime, duration_ms,
|
||||
artwork_url, state, remote_url, stamp, stamp),
|
||||
)
|
||||
self._db.commit()
|
||||
return media_id
|
||||
|
||||
def update_media(self, media_id: str, **fields) -> None:
|
||||
if not fields:
|
||||
return
|
||||
fields["updated_at"] = now()
|
||||
assignments = ", ".join(f"{key} = ?" for key in fields)
|
||||
self._db.execute(
|
||||
f"UPDATE media SET {assignments} WHERE id = ?", (*fields.values(), media_id)
|
||||
)
|
||||
self._db.commit()
|
||||
|
||||
def media(self, media_id: str) -> Media | None:
|
||||
row = self._db.execute("SELECT * FROM media WHERE id = ?", (media_id,)).fetchone()
|
||||
return Media(**dict(row)) if row else None
|
||||
|
||||
def media_since(self, since: int = 0, state: str | None = "ready") -> list[Media]:
|
||||
query = "SELECT * FROM media WHERE updated_at > ?"
|
||||
params: list = [since]
|
||||
if state:
|
||||
query += " AND state = ?"
|
||||
params.append(state)
|
||||
query += " ORDER BY updated_at ASC"
|
||||
return [Media(**dict(r)) for r in self._db.execute(query, params)]
|
||||
|
||||
def pending_downloads(self) -> list[Media]:
|
||||
return [
|
||||
Media(**dict(r))
|
||||
for r in self._db.execute(
|
||||
"SELECT * FROM media WHERE state IN ('pending','failed') AND remote_url IS NOT NULL "
|
||||
"ORDER BY created_at ASC LIMIT 20"
|
||||
)
|
||||
]
|
||||
|
||||
def delete_media(self, media_id: str) -> Media | None:
|
||||
item = self.media(media_id)
|
||||
if item:
|
||||
self._db.execute("DELETE FROM media WHERE id = ?", (media_id,))
|
||||
self._db.commit()
|
||||
return item
|
||||
|
||||
# --- Guets ----------------------------------------------------------
|
||||
|
||||
def add_watch(self, query: str) -> str:
|
||||
watch_id = uuid.uuid4().hex
|
||||
self._db.execute(
|
||||
"INSERT INTO watches (id,query,created_at) VALUES (?,?,?)",
|
||||
(watch_id, query.strip(), now()),
|
||||
)
|
||||
self._db.commit()
|
||||
return watch_id
|
||||
|
||||
def watches(self) -> list[dict]:
|
||||
return [dict(r) for r in self._db.execute("SELECT * FROM watches ORDER BY created_at DESC")]
|
||||
|
||||
def pending_watches(self) -> list[dict]:
|
||||
return [dict(r) for r in self._db.execute("SELECT * FROM watches WHERE found_at IS NULL")]
|
||||
|
||||
def mark_watch_found(self, watch_id: str, station: str, title: str) -> None:
|
||||
self._db.execute(
|
||||
"UPDATE watches SET found_at=?, found_on=?, found_title=? WHERE id=?",
|
||||
(now(), station, title, watch_id),
|
||||
)
|
||||
self._db.commit()
|
||||
|
||||
def mark_watches_scanned(self, ids: list[str]) -> None:
|
||||
if not ids:
|
||||
return
|
||||
stamp = now()
|
||||
self._db.executemany(
|
||||
"UPDATE watches SET scans = scans + 1, last_scan = ? WHERE id = ?",
|
||||
[(stamp, i) for i in ids],
|
||||
)
|
||||
self._db.commit()
|
||||
|
||||
def delete_watch(self, watch_id: str) -> None:
|
||||
self._db.execute("DELETE FROM watches WHERE id = ?", (watch_id,))
|
||||
self._db.commit()
|
||||
@@ -0,0 +1,267 @@
|
||||
<!doctype html>
|
||||
<html lang="fr">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Noxely — serveur</title>
|
||||
<style>
|
||||
:root { --bg:#faf7ff; --card:#fff; --ink:#1c1b22; --muted:#5c5670; --accent:#6b4eff;
|
||||
--mint:#11a894; --line:#e6e0f0; }
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root { --bg:#120d1c; --card:#1b1033; --ink:#e8e0f0; --muted:#a79fbb; --line:#2f2743; }
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
body { margin:0; background:var(--bg); color:var(--ink); font:15px/1.5 system-ui,sans-serif; }
|
||||
header { padding:18px 20px; border-bottom:1px solid var(--line); display:flex;
|
||||
align-items:center; gap:12px; flex-wrap:wrap; }
|
||||
h1 { font-size:19px; margin:0; }
|
||||
main { max-width:920px; margin:0 auto; padding:20px; }
|
||||
.tabs { display:flex; gap:6px; flex-wrap:wrap; margin-bottom:18px; }
|
||||
.tabs button { background:none; border:1px solid var(--line); border-radius:999px;
|
||||
padding:7px 15px; color:var(--muted); cursor:pointer; font-size:14px; }
|
||||
.tabs button[aria-selected="true"] { background:var(--accent); border-color:var(--accent); color:#fff; }
|
||||
.card { background:var(--card); border:1px solid var(--line); border-radius:14px;
|
||||
padding:12px 14px; margin-bottom:8px; display:flex; gap:12px; align-items:center; }
|
||||
.card .grow { flex:1; min-width:0; }
|
||||
.card b { display:block; overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
|
||||
.card small { color:var(--muted); }
|
||||
input, select { padding:9px 12px; border:1px solid var(--line); border-radius:10px;
|
||||
background:var(--card); color:var(--ink); font-size:14px; }
|
||||
input[type=text] { flex:1; min-width:160px; }
|
||||
button.go { background:var(--accent); color:#fff; border:none; border-radius:10px;
|
||||
padding:9px 16px; cursor:pointer; font-size:14px; }
|
||||
button.ghost { background:none; border:1px solid var(--line); border-radius:10px;
|
||||
padding:7px 12px; cursor:pointer; color:var(--muted); }
|
||||
.row { display:flex; gap:8px; margin-bottom:14px; flex-wrap:wrap; }
|
||||
.pill { font-size:12px; padding:3px 9px; border-radius:999px; background:var(--line); color:var(--muted); }
|
||||
.pill.ok { background:rgba(17,168,148,.16); color:var(--mint); }
|
||||
.pill.warn { background:rgba(224,98,155,.16); color:#e0629b; }
|
||||
.empty { color:var(--muted); text-align:center; padding:40px 10px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<h1>🎧 Noxely — serveur</h1>
|
||||
<span id="status" class="pill">…</span>
|
||||
<span class="grow" style="flex:1"></span>
|
||||
<input type="password" id="token" placeholder="Jeton" style="width:190px">
|
||||
<button class="ghost" onclick="saveToken()">Enregistrer</button>
|
||||
</header>
|
||||
<main>
|
||||
<div class="tabs">
|
||||
<button onclick="show('search')" aria-selected="true" id="t-search">🔎 Rechercher</button>
|
||||
<button onclick="show('watch')" aria-selected="false" id="t-watch">📡 Guets</button>
|
||||
<button onclick="show('queue')" aria-selected="false" id="t-queue">⏳ File</button>
|
||||
<button onclick="show('library')" aria-selected="false" id="t-library">📚 Bibliothèque</button>
|
||||
</div>
|
||||
|
||||
<section id="p-search">
|
||||
<div class="row">
|
||||
<select id="catalog"></select>
|
||||
<input type="text" id="q" placeholder="Titre, artiste, album…" onkeydown="if(event.key==='Enter')doSearch()">
|
||||
<button class="go" onclick="doSearch()">Chercher</button>
|
||||
</div>
|
||||
<div id="results"></div>
|
||||
</section>
|
||||
|
||||
<section id="p-watch" hidden>
|
||||
<div class="row">
|
||||
<input type="text" id="watchQuery" placeholder="Morceau à guetter sur toutes les stations"
|
||||
onkeydown="if(event.key==='Enter')addWatch()">
|
||||
<button class="go" onclick="addWatch()">Guetter</button>
|
||||
<button class="ghost" onclick="sweep()">Balayer maintenant</button>
|
||||
</div>
|
||||
<div id="watches"></div>
|
||||
</section>
|
||||
|
||||
<section id="p-queue" hidden><div id="queue"></div></section>
|
||||
<section id="p-library" hidden><div id="library"></div></section>
|
||||
</main>
|
||||
|
||||
<script>
|
||||
const $ = (id) => document.getElementById(id);
|
||||
let token = localStorage.getItem('noxely-token') || '';
|
||||
$('token').value = token;
|
||||
|
||||
function saveToken() {
|
||||
token = $('token').value.trim();
|
||||
localStorage.setItem('noxely-token', token);
|
||||
// Consultée depuis la machine du serveur, la page se remplit seule.
|
||||
(async () => {
|
||||
if (!token) {
|
||||
try {
|
||||
const { token: found } = await fetch('/api/bootstrap').then((r) => r.ok ? r.json() : {});
|
||||
if (found) { token = found; $('token').value = found; localStorage.setItem('noxely-token', found); }
|
||||
} catch {}
|
||||
}
|
||||
refreshAll();
|
||||
})();
|
||||
}
|
||||
|
||||
async function api(path, options = {}) {
|
||||
const response = await fetch('/api' + path, {
|
||||
...options,
|
||||
headers: { 'X-Noxely-Token': token, 'Content-Type': 'application/json', ...(options.headers || {}) },
|
||||
});
|
||||
if (!response.ok) throw new Error((await response.json().catch(() => ({}))).detail || response.status);
|
||||
return response.json();
|
||||
}
|
||||
|
||||
function show(name) {
|
||||
for (const tab of ['search', 'watch', 'queue', 'library']) {
|
||||
$('p-' + tab).hidden = tab !== name;
|
||||
$('t-' + tab).setAttribute('aria-selected', String(tab === name));
|
||||
}
|
||||
if (name === 'queue') loadQueue();
|
||||
if (name === 'library') loadLibrary();
|
||||
if (name === 'watch') loadWatches();
|
||||
}
|
||||
|
||||
async function refreshAll() {
|
||||
try {
|
||||
const health = await fetch('/api/health').then((r) => r.json());
|
||||
$('status').textContent = health.name + ' ' + health.version;
|
||||
$('status').className = 'pill ok';
|
||||
} catch { $('status').textContent = 'injoignable'; $('status').className = 'pill warn'; }
|
||||
try {
|
||||
const { catalogs } = await api('/catalogs');
|
||||
$('catalog').innerHTML = Object.entries(catalogs)
|
||||
.map(([id, label]) => `<option value="${id}">${label}</option>`).join('');
|
||||
} catch {}
|
||||
}
|
||||
|
||||
async function doSearch() {
|
||||
const q = $('q').value.trim(); if (!q) return;
|
||||
$('results').innerHTML = '<div class="empty">Recherche…</div>';
|
||||
try {
|
||||
const { results } = await api(`/search?catalog=${$('catalog').value}&q=${encodeURIComponent(q)}`);
|
||||
$('results').innerHTML = results.length ? results.map((r, i) => `
|
||||
<div class="card">
|
||||
<div class="grow"><b>${esc(r.title)}</b><small>${
|
||||
r.catalog ? '<span class="pill">' + esc(r.catalog) + '</span> ' : ''}${
|
||||
esc(r.artist)} · ${r.kind}${
|
||||
r.size ? ' · ~' + (r.size / 1e6).toFixed(0) + ' Mo' : ''}</small></div>
|
||||
<button class="ghost" onclick='openTracks(${i})'>Pistes</button>
|
||||
</div>`).join('') : '<div class="empty">Aucun résultat</div>';
|
||||
window._results = results;
|
||||
} catch (e) { $('results').innerHTML = `<div class="empty">${esc(String(e))}</div>`; }
|
||||
}
|
||||
|
||||
async function openTracks(index) {
|
||||
const item = window._results[index];
|
||||
$('results').innerHTML = '<div class="empty">Lecture des pistes…</div>';
|
||||
try {
|
||||
// En recherche « toutes sources », c'est le catalogue du résultat qui compte, pas celui du menu.
|
||||
const catalog = item.catalog || $('catalog').value;
|
||||
const { tracks } = await api(`/tracks?catalog=${catalog}&id=${
|
||||
encodeURIComponent(item.id)}&title=${encodeURIComponent(item.title)}&kind=${item.kind}`);
|
||||
window._catalog = catalog;
|
||||
window._tracks = tracks;
|
||||
const total = tracks.reduce((sum, t) => sum + (t.size || 0), 0);
|
||||
$('results').innerHTML = `
|
||||
<div class="row"><button class="ghost" onclick="doSearch()">← Résultats</button>
|
||||
<button class="go" onclick="queueAll()">Tout mettre en file${
|
||||
total ? ' (' + (total / 1e6).toFixed(0) + ' Mo)' : ''}</button></div>` +
|
||||
tracks.map((t, i) => `
|
||||
<div class="card">
|
||||
<div class="grow"><b>${esc(t.title)}</b><small>${
|
||||
t.size ? (t.size / 1e6).toFixed(1) + ' Mo' : 'taille inconnue'}</small></div>
|
||||
<button class="ghost" onclick="queueOne(${i})">Mettre en file</button>
|
||||
</div>`).join('');
|
||||
} catch (e) { $('results').innerHTML = `<div class="empty">${esc(String(e))}</div>`; }
|
||||
}
|
||||
|
||||
const queueTracks = async (tracks) => {
|
||||
const { queued } = await api('/queue', { method: 'POST',
|
||||
body: JSON.stringify({ tracks, catalog: window._catalog || $('catalog').value }) });
|
||||
alert(queued + ' piste(s) en file de téléchargement');
|
||||
};
|
||||
const queueOne = (i) => queueTracks([window._tracks[i]]);
|
||||
const queueAll = () => queueTracks(window._tracks);
|
||||
|
||||
async function loadQueue() {
|
||||
try {
|
||||
const { items } = await api('/queue');
|
||||
$('queue').innerHTML = items.length ? items.map((m) => `
|
||||
<div class="card"><div class="grow"><b>${esc(m.title)}</b>
|
||||
<small>${esc(m.artist)} · ${m.state}${m.error ? ' — ' + esc(m.error) : ''}</small></div></div>`).join('')
|
||||
: '<div class="empty">File vide</div>';
|
||||
} catch (e) { $('queue').innerHTML = `<div class="empty">${esc(String(e))}</div>`; }
|
||||
}
|
||||
|
||||
async function loadLibrary() {
|
||||
try {
|
||||
const { items } = await api('/library?since=0');
|
||||
$('library').innerHTML = items.length ? items.map((m) => `
|
||||
<div class="card"><div class="grow"><b>${esc(m.title)}</b>
|
||||
<small>${esc(m.artist)} · ${(m.size / 1e6).toFixed(1)} Mo · ${esc(m.source)}</small></div>
|
||||
<button class="ghost" onclick="removeMedia('${m.id}')">Supprimer</button></div>`).join('')
|
||||
: '<div class="empty">Rien de prêt pour le mobile</div>';
|
||||
} catch (e) { $('library').innerHTML = `<div class="empty">${esc(String(e))}</div>`; }
|
||||
}
|
||||
|
||||
async function removeMedia(id) {
|
||||
if (!confirm('Supprimer définitivement ce média du serveur ?')) return;
|
||||
await api('/media/' + id, { method: 'DELETE' });
|
||||
loadLibrary();
|
||||
}
|
||||
|
||||
async function loadWatches() {
|
||||
try {
|
||||
const { watches, recording, sweeps, last_sweep: last } = await api('/watches');
|
||||
const scan = last && last.stations
|
||||
? `${sweeps} tour(s) · dernier : ${last.stations} stations, ${last.titles} titres lus`
|
||||
: 'en attente d\'un guet';
|
||||
$('watches').innerHTML =
|
||||
`<div class="card"><div class="grow"><b>Balayage continu</b><small>${esc(scan)}</small></div>
|
||||
<span class="pill ok">actif</span></div>` +
|
||||
(recording ? `<div class="card"><div class="grow">
|
||||
<b>Enregistrement en cours</b><small>${esc(recording)}</small></div></div>` : '') +
|
||||
(watches.length ? watches.map((w) => `
|
||||
<div class="card"><div class="grow"><b>${esc(w.query)}</b><small>${
|
||||
w.found_at ? 'capté sur ' + esc(w.found_on || '') + ' — ' + esc(w.found_title || '')
|
||||
: w.scans + ' balayage(s)'}</small></div>
|
||||
<span class="pill ${w.found_at ? 'ok' : ''}">${w.found_at ? 'trouvé' : 'en attente'}</span>
|
||||
<button class="ghost" onclick="removeWatch('${w.id}')">×</button></div>`).join('')
|
||||
: '<div class="empty">Aucun guet</div>');
|
||||
} catch (e) { $('watches').innerHTML = `<div class="empty">${esc(String(e))}</div>`; }
|
||||
}
|
||||
|
||||
async function addWatch() {
|
||||
const query = $('watchQuery').value.trim(); if (!query) return;
|
||||
await api('/watches', { method: 'POST', body: JSON.stringify({ query }) });
|
||||
$('watchQuery').value = '';
|
||||
loadWatches();
|
||||
}
|
||||
const removeWatch = async (id) => { await api('/watches/' + id, { method: 'DELETE' }); loadWatches(); };
|
||||
|
||||
async function sweep() {
|
||||
$('watches').innerHTML = '<div class="empty">Balayage des stations…</div>';
|
||||
try {
|
||||
const r = await api('/watches/sweep', { method: 'POST' });
|
||||
alert(r.matched ? `Trouvé sur ${r.station} : ${r.matched}`
|
||||
: `${r.stations} stations écoutées, ${r.titles} titres lus, rien de guetté`);
|
||||
} catch (e) { alert(String(e)); }
|
||||
loadWatches();
|
||||
}
|
||||
|
||||
const esc = (s) => String(s).replace(/[&<>"']/g, (c) =>
|
||||
({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[c]));
|
||||
|
||||
// Consultée depuis la machine du serveur, la page se remplit seule.
|
||||
(async () => {
|
||||
if (!token) {
|
||||
try {
|
||||
const { token: found } = await fetch('/api/bootstrap').then((r) => r.ok ? r.json() : {});
|
||||
if (found) { token = found; $('token').value = found; localStorage.setItem('noxely-token', found); }
|
||||
} catch {}
|
||||
}
|
||||
refreshAll();
|
||||
})();
|
||||
setInterval(() => {
|
||||
if (!$('p-queue').hidden) loadQueue();
|
||||
if (!$('p-watch').hidden) loadWatches();
|
||||
}, 5000);
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,188 @@
|
||||
"""Tâches de fond : file de téléchargement, guet radio et enregistrement.
|
||||
|
||||
C'est ici que se déplace la consommation qui pesait sur le téléphone : le serveur balaie les
|
||||
stations et télécharge, le mobile ne fait plus que récupérer des fichiers prêts.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from .config import Config
|
||||
from .sources import radio
|
||||
from .storage import Storage
|
||||
|
||||
UA = {"User-Agent": "Noxely/1.0 (https://git.tuxtech.fr/tuxgyver/Noxely)"}
|
||||
|
||||
|
||||
def safe_name(text: str, extension: str) -> str:
|
||||
cleaned = re.sub(r"[^\w \-.]", "", text).strip().replace(" ", "_")[:80] or "media"
|
||||
return f"{cleaned}.{extension}"
|
||||
|
||||
|
||||
class Downloader:
|
||||
"""Vide la file des médias en attente, un à la fois."""
|
||||
|
||||
def __init__(self, config: Config, storage: Storage) -> None:
|
||||
self.config = config
|
||||
self.storage = storage
|
||||
self.config.media_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
async def run_forever(self) -> None:
|
||||
while True:
|
||||
try:
|
||||
await self.drain()
|
||||
except Exception as error: # une panne ne doit pas arrêter la boucle
|
||||
print(f"[noxely] téléchargement : {error}")
|
||||
await asyncio.sleep(10)
|
||||
|
||||
async def drain(self) -> None:
|
||||
for item in self.storage.pending_downloads():
|
||||
await self.fetch(item.id, item.remote_url, item.filename)
|
||||
|
||||
async def fetch(self, media_id: str, url: str, filename: str) -> None:
|
||||
target = self.config.media_dir / filename
|
||||
partial = target.with_suffix(target.suffix + ".part")
|
||||
self.storage.update_media(media_id, state="downloading", error=None)
|
||||
try:
|
||||
async with httpx.AsyncClient(headers=UA, timeout=httpx.Timeout(120.0),
|
||||
follow_redirects=True) as client:
|
||||
async with client.stream("GET", url) as response:
|
||||
response.raise_for_status()
|
||||
with partial.open("wb") as handle:
|
||||
async for chunk in response.aiter_bytes(65536):
|
||||
handle.write(chunk)
|
||||
partial.replace(target)
|
||||
self.storage.update_media(media_id, state="ready", size=target.stat().st_size)
|
||||
except Exception as error:
|
||||
partial.unlink(missing_ok=True)
|
||||
self.storage.update_media(media_id, state="failed", error=str(error)[:200])
|
||||
|
||||
|
||||
class Watcher:
|
||||
"""Balaie les stations à la recherche des morceaux guettés, et les enregistre."""
|
||||
|
||||
def __init__(self, config: Config, storage: Storage) -> None:
|
||||
self.config = config
|
||||
self.storage = storage
|
||||
self.recording: str | None = None
|
||||
#: Dernier tour de balayage, pour que l'interface montre que ça tourne.
|
||||
self.last_sweep: dict = {}
|
||||
self.sweeps = 0
|
||||
|
||||
async def run_forever(self) -> None:
|
||||
"""Balaie en continu : un tour enchaîne sur le suivant sans attendre.
|
||||
|
||||
Une sonde ne lit que le premier bloc de métadonnées puis referme — quelques dizaines de
|
||||
kilo-octets par station. C'est ce qui rend le balayage permanent tenable, là où garder
|
||||
quarante flux ouverts consommerait plusieurs mégabits par seconde en continu.
|
||||
"""
|
||||
while True:
|
||||
try:
|
||||
if self.storage.pending_watches():
|
||||
self.last_sweep = await self.sweep()
|
||||
self.sweeps += 1
|
||||
else:
|
||||
# Rien à guetter : inutile de solliciter l'annuaire.
|
||||
await asyncio.sleep(30)
|
||||
continue
|
||||
except Exception as error:
|
||||
print(f"[noxely] guet : {error}")
|
||||
await asyncio.sleep(30)
|
||||
continue
|
||||
await asyncio.sleep(self.config.watch_interval)
|
||||
|
||||
async def sweep(self) -> dict:
|
||||
pending = self.storage.pending_watches()
|
||||
if not pending:
|
||||
return {"stations": 0, "titles": 0, "at": _now(), "matched": None}
|
||||
|
||||
found = await radio.stations(limit=self.config.watch_stations)
|
||||
titles = 0
|
||||
for group in _chunks(found, self.config.watch_concurrency):
|
||||
heard = await asyncio.gather(*(radio.now_playing(s["url"]) for s in group))
|
||||
for station, title in zip(group, heard):
|
||||
if not title:
|
||||
continue
|
||||
titles += 1
|
||||
hit = next((w for w in pending if w["query"].lower() in title.lower()), None)
|
||||
if not hit:
|
||||
continue
|
||||
if self.recording is None:
|
||||
asyncio.create_task(self.record(station, title, hit["query"]))
|
||||
self.storage.mark_watch_found(hit["id"], station["name"], title)
|
||||
self.storage.mark_watches_scanned([w["id"] for w in pending])
|
||||
return {"stations": len(found), "titles": titles, "at": _now(),
|
||||
"matched": title, "station": station["name"]}
|
||||
|
||||
self.storage.mark_watches_scanned([w["id"] for w in pending])
|
||||
return {"stations": len(found), "titles": titles, "at": _now(), "matched": None}
|
||||
|
||||
async def record(self, station: dict, title: str, trigger: str) -> None:
|
||||
"""Enregistre jusqu'à ce que la station annonce un autre morceau."""
|
||||
self.recording = station["name"]
|
||||
extension = "aac" if station["codec"].upper().startswith("AAC") else "mp3"
|
||||
filename = safe_name(title, extension)
|
||||
media_id = self.storage.add_media(
|
||||
title=title, artist=station["name"], source="radio", origin=station["name"],
|
||||
filename=filename, remote_url=None,
|
||||
mime="audio/aac" if extension == "aac" else "audio/mpeg", state="downloading",
|
||||
)
|
||||
target = self.config.media_dir / filename
|
||||
headers = {**UA, "Icy-MetaData": "1"}
|
||||
written = 0
|
||||
try:
|
||||
async with httpx.AsyncClient(headers=headers, timeout=httpx.Timeout(60.0)) as client:
|
||||
async with client.stream("GET", station["url"]) as response:
|
||||
response.raise_for_status()
|
||||
interval = int(response.headers.get("icy-metaint") or 0)
|
||||
with target.open("wb") as handle:
|
||||
written = await self._copy(response, handle, interval, title)
|
||||
self.storage.update_media(media_id, state="ready", size=written)
|
||||
except Exception as error:
|
||||
self.storage.update_media(media_id, state="failed", error=str(error)[:200])
|
||||
finally:
|
||||
self.recording = None
|
||||
|
||||
async def _copy(self, response, handle, interval: int, wanted: str) -> int:
|
||||
"""Retire les blocs ICY du flux : laissés dedans, ils rendraient le fichier illisible."""
|
||||
if interval <= 0:
|
||||
total = 0
|
||||
async for chunk in response.aiter_bytes(32768):
|
||||
handle.write(chunk)
|
||||
total += len(chunk)
|
||||
if total > 40_000_000: # garde-fou : 40 Mo sans annonce de fin
|
||||
break
|
||||
return total
|
||||
|
||||
buffer = bytearray()
|
||||
total = 0
|
||||
async for chunk in response.aiter_bytes(32768):
|
||||
buffer.extend(chunk)
|
||||
while len(buffer) >= interval + 1:
|
||||
length = buffer[interval] * 16
|
||||
if len(buffer) < interval + 1 + length:
|
||||
break
|
||||
handle.write(bytes(buffer[:interval]))
|
||||
total += interval
|
||||
meta = bytes(buffer[interval + 1:interval + 1 + length]).decode("utf-8", "replace")
|
||||
del buffer[:interval + 1 + length]
|
||||
announced = radio.STREAM_TITLE.search(meta)
|
||||
if announced and announced.group(1).strip() and announced.group(1).strip() != wanted:
|
||||
return total # le morceau est terminé
|
||||
if total > 60_000_000:
|
||||
return total
|
||||
return total
|
||||
|
||||
|
||||
def _now() -> int:
|
||||
return int(time.time() * 1000)
|
||||
|
||||
|
||||
def _chunks(items: list, size: int):
|
||||
for index in range(0, len(items), size):
|
||||
yield items[index:index + size]
|
||||
@@ -0,0 +1,3 @@
|
||||
fastapi>=0.110.0
|
||||
uvicorn[standard]>=0.29.0
|
||||
httpx>=0.27.0
|
||||
Reference in new issue
Block a user