You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
spike 87bcc2f559 update 2 months ago
__pycache__ update 2 months ago
templates initial 2 months ago
.gitignore update 2 months ago
LICENSE.txt initial 2 months ago
Pipfile initial 2 months ago
Pipfile.lock initial 2 months ago
README.md initial 2 months ago
database_connection.py initial 2 months ago
decrypt_attachment.py update 2 months ago
download_images.py update 2 months ago
export_messages.py update 2 months ago
freeze.txt update 2 months ago
import_messages.py initial 2 months ago
list_rooms.py initial 2 months ago
matrix_connection.py initial 2 months ago
schema.py update 2 months ago
setup.cfg initial 2 months ago

README.md

Matrix Archive Tools

Import messages from a matrix.org room, for research, archival, and preservation.

Developed at Dinacon 2018, for use by the documentation team.

Use this responsibly and ethically. Don't re-publish people's messages without their knowledge and consent.

Setup

Install Pipenv. Run pipenv install.

Set these environment variables: MATRIX_USER, MATRIX_PASSWORD, MATRIX_ROOM_IDS.

MATRIX_ROOM_IDS should be a comma-separated list of Matrix room IDs (or a single id). Run pipenv run list_rooms.py to list the room ids.

Set MONGODB_URI to a MongoDB connection URL, or install a local MongoDB instance.

Usage

Import Messages

pipenv run import imports the messages into the database.

Export Messages

pipenv run export filename.html exports a text, HTML, JSON, or YAML file, depending on the name of filename.html. The file contains links to the image download URLs on the Matrix server.

Download Images

pipenv run download_images.py downloads all the thumbnail images in the database into a download directory (default thumbnails), skipping images that have already been downloaded.

Use the --no-thumbnails option to download full size images instead of thumbnails. In this case, the default directory is images instead of thumbnails.

References

Matrix Client-Server API

License

MIT