From 4232e41d6b9be7f4a44ac55342d5644521c9b09a Mon Sep 17 00:00:00 2001 From: David Janowski Date: Thu, 22 Jun 2023 09:35:13 +0200 Subject: [PATCH] improved documentation --- README.MD | 120 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 118 insertions(+), 2 deletions(-) diff --git a/README.MD b/README.MD index b1c15df..e31b0d3 100644 --- a/README.MD +++ b/README.MD @@ -1,2 +1,118 @@ -## TODO: --[ ] Que & Channels +# e621 to Graph + +Scrapes users Favorite Posts and Upload them to a graph based database. + +## Prerequisites + +Before running the program, ensure that you have the following installed on your system: + +- Go (version 1.20.2 or higher) +- Docker (optional, if you plan to use Docker) + +## Installation + +To install the project, follow these steps: + +1. Clone the repository: + + ```shell + git clone https://git.dragse.it/fenpaws/e621-to-graph + ``` + +2. Change into the project's directory: + + ```shell + cd e621-to-graph + ``` + +3. Install the required dependencies: + + ```shell + go mod download + go mod verify + ``` + +## Setting Up Environment Variables + +The program requires certain environment variables to be set. +```plaintext +E621_API_KEY= +E621_USERNAME= +DB_TYPE=neo4j +DB_URL= +NEO4J_PASSWORD= +NEO4J_USERNAME= +``` + +## Running the Program + +To run the program, execute the following command: + +```shell +go run main.go +``` + +## Building the Program + +If you want to build the program into an executable binary, use the following command: + +```shell +go build +``` + +## Docker Support + +The program can also be run using Docker. To use Docker, follow these steps: + +1. Create a .env file in the project's root directory with the following content: + + ````plaintext + E621_API_KEY= + E621_USERNAME= + DB_TYPE=neo4j + DB_URL= + NEO4J_PASSWORD= + NEO4J_USERNAME= + ```` + +2. Build the Docker image: + + ```shell + docker compose build + ``` + +3. Run the Docker container: + + ```shell + docker compose up -d + ``` + +## Database Support + +The program supports the following databases. + +### Neo4j + +To improve performance, it is recommended to create indices on different nodes in Neo4j. + +Run the following commands to create the required indices: + +```shell +neo4j$ CREATE INDEX tagIndex FOR (t:e621Tag) ON (t.e621Tag); +neo4j$ CREATE INDEX postIndex FOR (p:e621Post) ON (p.e621PostID); +neo4j$ CREATE INDEX sourceUrlIndex FOR (s:Source) ON (s.URL); +neo4j$ CREATE INDEX userIdIndex FOR (u:e621User) ON (u.e621ID); +``` + +### Memgraph: + +To improve performance, it is recommended to create indices on different nodes in Memgraph. + +Run the following commands to create the required indices: + +```shell +memgraph> CREATE INDEX ON :e621Tag(e621Tag); +memgraph> CREATE INDEX ON :e621Post(e621PostID); +memgraph> CREATE INDEX ON :Source(URL); +memgraph> CREATE INDEX ON :e621User(e621ID); +```