Juridical Worker

Worker responsible for web scraping and identifying legal processes from the Projudi system for the Justice Tribunal of Brazil.

Environment settings local

.NET

• Use dotnet user-secrets

1. Get project folder legal process worker:

src/Juridical.LegalProcess.Worker

2. Create secrets:

dotnet user-secrets set "LEGAL_PROCESS_USER" "YOUR_SECRET"
dotnet user-secrets set "LEGAL_PROCESS_PASSWORD" "YOUR_SECRET"

1. Get project folder message worker:

src/Juridical.Message.Worker

2. Create secrets:

dotnet user-secrets set "MESSAGE_SERVICE_API_TOKEN" "YOUR_SECRET"
dotnet user-secrets set "MESSAGE_SERVICE_FROM" "YOUR_SECRET"
dotnet user-secrets set "MESSAGE_SERVICE_TO" "YOUR_SECRET"

Docker

• Create .env file

PROJECT_ID=juridical-test
PUBSUB_EMULATOR_HOST=127.0.0.1:8085
WEB_DRIVER_URI=http://juridical-selenium:4444/wd/hub
LEGAL_PROCESS_USER=YOUR_SECRET
LEGAL_PROCESS_PASSWORD=YOUR_SECRET
MESSAGE_SERVICE_API_TOKEN=YOUR_SECRET
MESSAGE_SERVICE_FROM=YOUR_SECRET
MESSAGE_SERVICE_TO=YOUR_SECRET

Instructions for run project

Pub/Sub Emulator

1. Run pub/sub emulator:

cd emulators/ && docker-compose up -d

2. Publish message:

docker exec -it juridical-pubsub-emulator /bin/bash

python3 /root/bin/pubsub-client.py publish juridical-test juridical.legal-process.resulted '{
  "specversion": "1.0",
  "id": "542204ea-76c7-4b38-a35d-55440bfa3b6a",
  "type": "Juridical.Core.Events.LegalProcessEvent",
  "source": "juridical-legal-process-worker",
  "datacontenttype": "application/json",
  "time": "2023-06-09T14:58:21.6717314-03:00",
  "data": "{\"processCount\":1}"
}'

.NET

1. Run selenium:

docker run -d -p 4444:4444 -p 7900:7900 --shm-size="2g" -e VNC_NO_PASSWORD=1 --name selenium selenium/standalone-chrome:123.0

2. Run projects:

cd src/Juridical.LegalProcess.Worker && dotnet watch run

cd src/Juridical.Message.Worker && dotnet watch run

Docker

• Run project

docker-compose up -d

Push images (optional)

• Create Container Registry (GCP)

1. Configure auth GCP CLI login:

gcloud auth login

2. Configure auth configure docker:

gcloud auth configure-docker

3. Push images for private registry:

docker build \
  -f ./src/Juridical.LegalProcess.Worker/Dockerfile \
  -t juridical/juridical-legal-process-worker:v1 \
  ./src/ &&
docker tag juridical/juridical-legal-process-worker:v1 us-east1-docker.pkg.dev/$PROJECT_ID/juridical/juridical-legal-process-worker:v1 &&
docker push us-east1-docker.pkg.dev/$PROJECT_ID/juridical/juridical-legal-process-worker:v1

docker build \
  -f ./src/Juridical.Message.Worker/Dockerfile \
  -t juridical/juridical-message-worker:v1 \
  ./src/ &&
docker tag juridical/juridical-message-worker:v1 us-east1-docker.pkg.dev/$PROJECT_ID/juridical/juridical-message-worker:v1 &&
docker push us-east1-docker.pkg.dev/$PROJECT_ID/juridical/juridical-message-worker:v1

Infrastructure

Terraform

• Create service account from GCP

1. Create service account:

gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME \
  --display-name "$SERVICE_ACCOUNT_DISPLAY_NAME" --project $PROJECT_ID

2. Get service account email:

gcloud iam service-accounts list

3. Create credentials key:

# SERVICE_ACCOUNT_CREDENTIALS=~/.config/gcloud/CREDENTIALS_FILE_NAME.json

gcloud iam service-accounts keys create $SERVICE_ACCOUNT_CREDENTIALS \
  --iam-account $SERVICE_ACCOUNT_EMAIL

4. Add policy permissions:

gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/storage.admin
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/artifactregistry.admin
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/container.admin
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/iam.serviceAccountUser
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/viewer
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/pubsub.admin

• Run local infrastructure

1. Install Terraform and create GOOGLE_CREDENTIALS variable:

export GOOGLE_CREDENTIALS=~/.config/gcloud/CREDENTIALS_FILE_NAME.json

2. Execute init:

cd infra/ && terraform init

3. Execute apply:

terraform apply \
  -var="project_id=$PROJECT_ID" \
  -var="service_account=$SERVICE_ACCOUNT_EMAIL"

• (Optional) Create remote backend bucket in Cloud Storage:

1. Create bucket:

gsutil mb -p $PROJECT_ID -l $LOCATION -b on gs://$BUCKET_NAME

Deploy

GitHub Actions

• Create service account from GCP

1. Create service account:

gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME \
  --display-name "$SERVICE_ACCOUNT_DISPLAY_NAME" --project $PROJECT_ID

2. Enable IAM Credentials:

gcloud services enable iamcredentials.googleapis.com --project $PROJECT_ID

3. Get service account email:

gcloud iam service-accounts list

4. Add policy permissions:

gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/container.admin
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/storage.admin
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/artifactregistry.admin
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/container.clusterViewer
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/logging.logWriter
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/monitoring.metricWriter
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/stackdriver.resourceMetadata.writer
gcloud projects add-iam-policy-binding $PROJECT_ID \
	--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
	--role=roles/pubsub.admin

• Enabling keyless authentication from GitHub Actions GCP

1. Create Workload Identity pool:

gcloud iam workload-identity-pools create "$POOL_NAME" \
  --project="$PROJECT_ID" \
  --location="global" \
  --display-name="$POOL_DISPLAY_NAME"

2. Get Workload Identity Id:

gcloud iam workload-identity-pools describe "$POOL_NAME" \
  --project="$PROJECT_ID" \
  --location="global" \
  --format="value(name)"

3. Create Workload Identity GitHub provider:

gcloud iam workload-identity-pools providers create-oidc "$PROVIDER_NAME" \
  --project="$PROJECT_ID" \
  --location="global" \
  --workload-identity-pool="$POOL_NAME" \
  --display-name="$PROVIDER_DISPLAY_NAME" \
  --attribute-mapping="google.subject=assertion.sub,attribute.actor=assertion.actor,attribute.repository=assertion.repository" \
  --issuer-uri="https://token.actions.githubusercontent.com

4. Create authentications from the Workload Identity provider:

gcloud iam service-accounts add-iam-policy-binding "$SERVICE_ACCOUNT_EMAIL" \
  --project="$PROJECT_ID" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/$WORKLOAD_IDENTITY_POOL_ID/attribute.repository/$GITHUB_USER/$GITHUB_REPOSITORY"

5. Get Workload Identity Provider resource name:

gcloud iam workload-identity-pools providers describe "$PROVIDER_NAME" \
  --project="$PROJECT_ID" \
  --location="global" \
  --workload-identity-pool="$POOL_NAME" \
  --format="value(name)"