56 lines
3.4 KiB
Markdown
56 lines
3.4 KiB
Markdown
# Email Unsubscribe Links Scraper
|
|
|
|
This is a simple Python web application that logs into an email server, scans the inbox for unsubscribe links, and displays the results in a table. It also provides the option to move selected emails to the trash.
|
|
|
|
## Disclaimer
|
|
This App is still in development
|
|
|
|
## How to use
|
|
|
|
1. Clone this repository.
|
|
2. Build the Docker image:
|
|
```docker build -t email-unsubscribe-links-scraper .```
|
|
3. Run the Docker container:
|
|
```docker run -p 5000:5000 -e EMAIL_USERNAME=your_email@example.com -e EMAIL_PASSWORD=your_email_password -e EMAIL_SERVER=your_email_server email-unsubscribe-links-scraper```
|
|
4. Access the application in your web browser at http://localhost:5000.
|
|
|
|
### For non-build-use:
|
|
Run following command: ```docker run -p 5000:5000 -e EMAIL_USERNAME=your_email@example.com -e EMAIL_PASSWORD=your_email_password -e EMAIL_SERVER=your_email_server rsclub22/email-unsubscribe-links-scraper```
|
|
|
|
## License
|
|
|
|
Copyright 2023 Philipp Wagner
|
|
|
|
This project is licensed under the GPLv3 License - see the [LICENSE](LICENSE) file for details.
|
|
|
|
## Betrieb
|
|
|
|
Konfiguration per Umgebung: `EMAIL_SERVER`, `EMAIL_PORT` (993), `EMAIL_USERNAME`, `EMAIL_PASSWORD`
|
|
(Sammelpostfach), optional `EMAIL_FOLDER` (`INBOX`) und `TRASH_FOLDER` (`Trash`).
|
|
Statt `EMAIL_PASSWORD` geht SSO (OAuth/XOAUTH2): `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`,
|
|
`OIDC_REDIRECT_URI` (`https://<host>/oidc/callback`), `TOKEN_KEY` (Fernet-Schlüssel) und `DATA_DIR` (`/data`).
|
|
Dann zeigt die Startseite „Connect mailbox“: einmal im privaten Fenster als technisches Konto des
|
|
Sammelpostfachs am SSO anmelden; das Refresh-Token liegt verschlüsselt in `DATA_DIR/oauth-token`.
|
|
Der Container startet gunicorn auf Port 5000 (kein Flask-Debugger) und gehört hinter einen
|
|
Reverse Proxy mit Anmeldung (bei Wagnersnetz Caddy `forward_auth` über Authelia).
|
|
Tests: `pytest tests`.
|
|
|
|
### Unsubscribe agent
|
|
|
|
With `DATA_DIR` set the app keeps one case per mail in `DATA_DIR/cases.db` and shows them under `/result`.
|
|
With `AGENT_URL` (the agent container from `agent/`, `http://<name>:8080`) a background run every
|
|
`AGENT_INTERVAL_MINUTES` (10) hands each new case to the agent: it opens the unsubscribe link in a sandboxed
|
|
browser, first checks for a confirmation without a model and otherwise lets a model operate the page with five
|
|
actions (click, select, check, insert the address, done). Successes are moved to the IMAP folder `DONE_FOLDER`
|
|
(`Erledigt`) and deleted after `RETENTION_DAYS` (30); everything else stays in the inbox with a reason.
|
|
"Run now" starts a run in the background and returns at once; the result page shows its outcome afterwards.
|
|
Further settings: `AGENT_CASE_TIMEOUT` (300), `AGENT_MAX_CASES` (10), `AGENT_ALLOWED_DOMAINS` (comma separated;
|
|
only addresses of these domains are ever typed into a page), `AGENT_SCHEDULER=0` (no background run).
|
|
`AGENT_ALLOWED_SENDERS` (comma separated) lists the forwarders whose mails become cases; mails from other
|
|
senders are ignored. With the agent on and an empty list nothing is processed.
|
|
|
|
The agent container needs `LLM_BASE_URL` (Open WebUI, `/api/chat/completions` is appended), `LLM_MODEL`,
|
|
`LLM_API_KEY`, optionally `MAX_STEPS` (20) and `CASE_TIMEOUT` (300). It must run in a network from which private
|
|
addresses are not reachable; the URL guard in `agent/guard.py` is only the first line of defence.
|
|
Tests: `pytest tests`, `cd agent && pytest tests` and `sh agent/tests/run-in-docker.sh` (browser tests).
|