Setup Instructions

Docker

You can easily run the complete erogaki project using our Docker images and our Docker Compose file (located here).

Please note that the complete erogaki project (as defined in the Docker Compose file) needs quite a lot of RAM (somewhere around 13 Gibibyte).

1. Step: Create a Models Docker Volume

The models aren't included with our releases, since we're unsure about their licensing.

Create a Docker volume for the models (this volume will be shared by multiple parts of the erogaki infrastructure):

docker volume create erogaki-models

Then get the following models and put them into the erogaki-models docker volume (you can find out the mountpoint using docker volume inspect erogaki-models):

  • hent-AI model 268
  • 09-11-2019 DCPv2 model

You can find a link to hent-AI model 268 here. And a link to 09-11-2019 DCPv2 model here.

Your erogaki-models volume should then have the following subdirectories:

  • hent-AI model 268/
  • 09-11-2019 DCPv2 model/

2. Step: Get the Files for Docker Compose

Clone https://github.com/erogaki-dev/erogaki to get all the relevant files.

3. Step: Provide a Discord Token

Provide a Discord token for erogaki-discord in erogaki-discord.env.

4. Step: Running the Erogaki Infrastructure

Finally you can run the whole erogaki project using:

docker-compose up

Design

Redis Flows

The following needs to be set:

redis-cli config set notify-keyspace-events AKE

(this can probably be more specific)

Flows

erogaki-discord: image request and receive

user message of type: !decensor manual bar

create image request

redis:

SET masked-images:uuid <image-data>
SUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
SUBSCRIBE __keyevent@0__:set errors:<uuid>
RPUSH decensor-requests:bar <uuid>

after keyevent decensored-images:<uuid>

Unsubscribe from keyevents, get the decensored image and delete all images.

redis:

UNSUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
UNSUBSCRIBE __keyevent@0__:set errors:<uuid>
GET decensored-images:uuid
DEL decensored-images:uuid
DEL masked-images:uuid

Send the received image to the user.

after keyevent errors:<uuid>

Unsubscribe from keyevents, get the error json and delete it and all images.

redis:

UNSUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
UNSUBSCRIBE __keyevent@0__:set errors:<uuid>
GET errors:uuid
DEL errors:uuid
DEL masked-images:uuid

Use the json to craft an error message for the user and send it.

user message of type: !decensor auto bar

create image request

redis:

SET censored-images:uuid <image-data>
SUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
SUBSCRIBE __keyevent@0__:set errors:<uuid>
RPUSH mask-requests:bar <uuid>

after keyevent decensored-images:<uuid>

Unsubscribe from keyevents, get the decensored image and delete all images.

redis:

UNSUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
UNSUBSCRIBE __keyevent@0__:set errors:<uuid>
GET decensored-images:uuid
DEL decensored-images:uuid
DEL masked-images:uuid
DEL censored-images:uuid

Send the received image to the user.

after keyevent errors:<uuid>

Unsubscribe from keyevents, get the error json and delete it and all images.

redis:

UNSUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
UNSUBSCRIBE __keyevent@0__:set errors:<uuid>
GET errors:uuid
DEL errors:uuid
DEL masked-images:uuid
DEL censored-images:uuid

Use the json to craft an error message for the user and send it.

user message of type: !decensor auto mosaic

create image request

redis:

SET censored-images:uuid <image-data>
SUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
SUBSCRIBE __keyevent@0__:set errors:<uuid>
RPUSH mask-requests:mosaic <uuid>

after keyevent decensored-images:<uuid>

Unsubscribe from keyevents, get the decensored image and delete all images.

redis:

UNSUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
UNSUBSCRIBE __keyevent@0__:set errors:<uuid>
GET decensored-images:uuid
DEL decensored-images:uuid
DEL masked-images:uuid
DEL censored-images:uuid

Send the received image to the user.

after keyevent errors:<uuid>

Unsubscribe from keyevents, get the error json and delete it and all images.

redis:

UNSUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
UNSUBSCRIBE __keyevent@0__:set errors:<uuid>
GET errors:uuid
DEL errors:uuid
DEL masked-images:uuid
DEL censored-images:uuid

Use the json to craft an error message for the user and send it.

user message of type: !decensor

create image request

redis:

SET censored-images:uuid <image-data>
SUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
SUBSCRIBE __keyevent@0__:set errors:<uuid>
RPUSH classify-requests <uuid>

after keyevent decensored-images:<uuid>

Unsubscribe from keyevents, get the decensored image and delete all images.

redis:

UNSUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
UNSUBSCRIBE __keyevent@0__:set errors:<uuid>
GET decensored-images:uuid
DEL decensored-images:uuid
DEL masked-images:uuid
DEL censored-images:uuid

Send the received image to the user.

after keyevent errors:<uuid>

Unsubscribe from keyevents, get the error json and delete it and all images.

redis:

UNSUBSCRIBE __keyevent@0__:set decensored-images:<uuid>
UNSUBSCRIBE __keyevent@0__:set errors:<uuid>
GET errors:uuid
DEL errors:uuid
DEL masked-images:uuid
DEL censored-images:uuid

Use the json to craft an error message for the user and send it.

erogaki-classify

wait for new image uuid

BLPOP classify-requests 0

returns:

1) key
2) value = uuid

get image data

redis:

GET censored-images:uuid

returns:

<image-data>

classify image

classify the image

if classification was successful: create detect request

if classified bar with most confidence

RPUSH mask-requests:bar <uuid>

if classified mosaic with most confidence

RPUSH mask-requests:mosaic <uuid>

if classification wasn't successful: return error

error:

{
    "component": "erogaki-classify",
    "instance": "<instance-name>",
    "name": "<error-name>",
    "description": "<error-description>"
}

redis:

SET errors:uuid error

erogaki-mask

wait for new image uuid

BLPOP mask-requests:bar|mosaic 0

returns:

1) key
2) value = uuid

get image data

redis:

GET censored-images:uuid

returns:

<image-data>

mask image

mask the image

if masking was successul: create decensor request

RPUSH decensor-requests:<key> <uuid>
SET masked-images:uuid <image-data>

if masking wasn't successful: return error

error:

{
    "component": "erogaki-mask",
    "instance": "<instance-name>",
    "name": "<error-name>",
    "description": "<error-description>"
}

redis:

SET errors:uuid error

DeepCreamPy-erogaki-wrapper

wait for new image uuid

BLPOP decensor-requests:bar|mosaic 0

returns:

1) key
2) value = uuid

get new image data

if key === bar

redis:

GET masked-images:uuid

returns:

<image-data>

if key === mosaic

redis:

GET masked-images:uuid
GET censored-images:uuid

returns:

<image-data> x2

process image/s

process the image/s

if processing was sucessful: return the image

redis:

SET decensored-images:uuid <image-data>

if processing wasn't sucessful: return an error

error:

{
    "component": "DeepCreamPy-erogaki-wrapper",
    "instance": "<instance-name>",
    "name": "<error-name>",
    "description": "<error-description>"
}

redis:

SET errors:uuid error

Redis Keys

Information Stores

  • censored-images:<uuid>
  • masked-images:<uuid>
  • decensored-images:<uuid>
  • errors:<uuid>

Queues

  • classify-requests
  • mask-requests:bar|mosaic
  • decensor-requests:bar|mosaic