ProcTHOR-Nav-Vis

This project is designed for visualization and demo video generation within PROCTHOR environments.

I originally created it to support presentations for my research paper, but since it may be useful to others, I’ve decided to make it public.

I. Installation

1. set environment

conda create -n procthor_nav_vis python=3.10 -y
conda activate procthor_nav_vis
pip install -r requirements.txt

2. download ai2thor binaries

bash set_binary.sh

3. download ProcTHOR-10K datasets

bash download_procthor_jsonl.sh

4. Convert jsonl files to json files

bash get_jsons.sh

II. Interact.py

python interact.py --arguments ...

Interact with the PROCTHOR environment or annotate coordinates and actions to script demonstration videos.

Control

• W / A / S / D — Move your agent

• Shift + WASD — Run

• H — Toggle mouse control

  Press H to enable mouse movement (free camera control).

  Press H again to lock the mouse and rotate the agent’s camera.

• L — Print the agent’s location information in the console

  You can use this data to generate script files.

• E — Display available actions for visible objects

• P — Terminate the program

• ← / → — Switch between different house environments (by index)

• Action Buttons (Tkinter Window) —

  After pressing E, a Tkinter window will display a list of available actions.

  Click any button to execute the corresponding action.

III. Record.py

python record.py --arguments ...

Replay scripts and record videos

camearas

• is_front : saving front, ego-centric video

• is_back : saving back video

• is_left : saving left video

• is_right : saving right video

• is_agent : saving agent video

Recording example

python record.py --script_txt scripts/example_script.txt --is_front --is_agent 

Example Scripts

scripts/example_script.txt

NAV x=4.9017 y=0.9010 z=1.8015 yaw=184.44 pitch=25.92
ACTION PickupObject objectId=CellPhone|surface|7|28
WAIT 0.5
NAV x=5.4067 y=0.9010 z=3.8827 yaw=6.96 pitch=28.92
NAV x=5.9052 y=0.9010 z=4.5145 yaw=44.52 pitch=12.96
NAV x=6.5516 y=0.9010 z=4.9942 yaw=137.40 pitch=41.64
WAIT 0.5
ACTION PutDown objectId=CellPhone|surface|7|28 receptacleObjectId=SideTable|7|2

Results

Setting	Front	Back	Left	Right
Agent View
Ego-centric