This repository has been archived on 2024-07-22. You can view files and clone it, but cannot push or open issues or pull requests.
e621-sdk-go/README.MD

119 lines
2.4 KiB
Plaintext
Raw Normal View History

2023-06-22 07:35:13 +00:00
# e621 to Graph
Scrapes users Favorite Posts and Upload them to a graph based database.
## Prerequisites
Before running the program, ensure that you have the following installed on your system:
- Go (version 1.20.2 or higher)
- Docker (optional, if you plan to use Docker)
## Installation
To install the project, follow these steps:
1. Clone the repository:
```shell
git clone https://git.dragse.it/fenpaws/e621-to-graph
```
2. Change into the project's directory:
```shell
cd e621-to-graph
```
3. Install the required dependencies:
```shell
go mod download
go mod verify
```
## Setting Up Environment Variables
The program requires certain environment variables to be set.
```plaintext
E621_API_KEY=
E621_USERNAME=
DB_TYPE=neo4j
DB_URL=
DB_PASSWORD=
DB_USERNAME=
2023-06-22 07:35:13 +00:00
```
## Running the Program
To run the program, execute the following command:
```shell
go run main.go
```
## Building the Program
If you want to build the program into an executable binary, use the following command:
```shell
go build
```
## Docker Support
The program can also be run using Docker. To use Docker, follow these steps:
1. Create a .env file in the project's root directory with the following content:
````plaintext
E621_API_KEY=
E621_USERNAME=
DB_TYPE=neo4j
DB_URL=
DB_PASSWORD=
DB_USERNAME=
2023-06-22 07:35:13 +00:00
````
2. Build the Docker image:
```shell
docker compose build
```
3. Run the Docker container:
```shell
docker compose up -d
```
## Database Support
The program supports the following databases.
### Neo4j
To improve performance, it is recommended to create indices on different nodes in Neo4j.
Run the following commands to create the required indices:
```shell
neo4j$ CREATE INDEX tagIndex FOR (t:e621Tag) ON (t.e621Tag);
neo4j$ CREATE INDEX postIndex FOR (p:e621Post) ON (p.e621PostID);
neo4j$ CREATE INDEX sourceUrlIndex FOR (s:Source) ON (s.URL);
neo4j$ CREATE INDEX userIdIndex FOR (u:e621User) ON (u.e621ID);
```
### Memgraph:
To improve performance, it is recommended to create indices on different nodes in Memgraph.
Run the following commands to create the required indices:
```shell
memgraph> CREATE INDEX ON :e621Tag(e621Tag);
memgraph> CREATE INDEX ON :e621Post(e621PostID);
memgraph> CREATE INDEX ON :Source(URL);
memgraph> CREATE INDEX ON :e621User(e621ID);
```