improved documentation

This commit is contained in:
David Janowski 2023-06-22 09:35:13 +02:00
parent 6d7fd4395d
commit 4232e41d6b

120
README.MD
View File

@ -1,2 +1,118 @@
## TODO:
-[ ] Que & Channels
# e621 to Graph
Scrapes users Favorite Posts and Upload them to a graph based database.
## Prerequisites
Before running the program, ensure that you have the following installed on your system:
- Go (version 1.20.2 or higher)
- Docker (optional, if you plan to use Docker)
## Installation
To install the project, follow these steps:
1. Clone the repository:
```shell
git clone https://git.dragse.it/fenpaws/e621-to-graph
```
2. Change into the project's directory:
```shell
cd e621-to-graph
```
3. Install the required dependencies:
```shell
go mod download
go mod verify
```
## Setting Up Environment Variables
The program requires certain environment variables to be set.
```plaintext
E621_API_KEY=
E621_USERNAME=
DB_TYPE=neo4j
DB_URL=
NEO4J_PASSWORD=
NEO4J_USERNAME=
```
## Running the Program
To run the program, execute the following command:
```shell
go run main.go
```
## Building the Program
If you want to build the program into an executable binary, use the following command:
```shell
go build
```
## Docker Support
The program can also be run using Docker. To use Docker, follow these steps:
1. Create a .env file in the project's root directory with the following content:
````plaintext
E621_API_KEY=
E621_USERNAME=
DB_TYPE=neo4j
DB_URL=
NEO4J_PASSWORD=
NEO4J_USERNAME=
````
2. Build the Docker image:
```shell
docker compose build
```
3. Run the Docker container:
```shell
docker compose up -d
```
## Database Support
The program supports the following databases.
### Neo4j
To improve performance, it is recommended to create indices on different nodes in Neo4j.
Run the following commands to create the required indices:
```shell
neo4j$ CREATE INDEX tagIndex FOR (t:e621Tag) ON (t.e621Tag);
neo4j$ CREATE INDEX postIndex FOR (p:e621Post) ON (p.e621PostID);
neo4j$ CREATE INDEX sourceUrlIndex FOR (s:Source) ON (s.URL);
neo4j$ CREATE INDEX userIdIndex FOR (u:e621User) ON (u.e621ID);
```
### Memgraph:
To improve performance, it is recommended to create indices on different nodes in Memgraph.
Run the following commands to create the required indices:
```shell
memgraph> CREATE INDEX ON :e621Tag(e621Tag);
memgraph> CREATE INDEX ON :e621Post(e621PostID);
memgraph> CREATE INDEX ON :Source(URL);
memgraph> CREATE INDEX ON :e621User(e621ID);
```