huawei-noah / smarts Goto Github PK

Move docker images to Docker Hub or other registry.

Duplicate of #59

Split from: #17

The intended behavior with this feature would be for boid agents to get created once per episode and stay alive regardless of all vehicles exiting the boid. For discussion: if we want to make this behavior optional, or the only option.

Benchmark,

1 agent, 50 SVs, loop scenario: 100 FPS
10 agents, 0 SVs, loop scenario, 25 FPS

Feel free to suggest a different benchmark than ^

Context:

davidrusu: Trying to run both agents and social vehicles together hides the performance hit from each individually.
Here's the curves of how increasing either value, slows our performance:

Tests were run on the loop scenario and Agents were instances of the following AgentSpec:

class Policy(AgentPolicy):
    def act(self, obs):
        return "keep_lane"

agent_spec = AgentSpec(
    interface=AgentInterface.from_type(AgentType.Laner, max_episode_steps=1000),
    policy_builder=Policy,
)

davidrusu: @jun Well, we can't directly control how many social vehicles are in the scenario, but here's a few episodes with a traffic flow of 30SV/min + 10 agents.

╭────────────────────┬────────────────────┬────────────────────┬────────────────────┬────────────────────┬╮
│            Episode │     Sim T / Wall T │        Total Steps │        Steps / Sec │       Scenario Map ││
├────────────────────┼────────────────────┼────────────────────┼────────────────────┼────────────────────┼┤
│              0/100 │               2.18 │                999 │              21.78 │               loop ││
│              1/100 │               1.32 │                999 │              13.23 │               loop ││
│              2/100 │               1.24 │                999 │              12.35 │               loop ││
│              3/100 │               1.59 │                999 │              15.87 │               loop ││

This is annoying

pybullet build time: Sep 20 2020 23:49:22
pybullet build time: Sep 20 2020 23:49:22
│              0/100 │               3.99 │                202 │              39.89 │            4lane_t │      basic.rou.xml │ 189590991351023167 │ 185.39 - Agent-007 │
pybullet build time: Sep 20 2020 23:49:22
pybullet build time: Sep 20 2020 23:49:22
pybullet build time: Sep 20 2020 23:49:22
│              1/100 │               4.72 │                202 │              47.25 │            4lane_t │      basic.rou.xml │ 189590991351023167 │ 185.39 - Agent-007 │
pybullet build time: Sep 20 2020 23:49:22
pybullet build time: Sep 20 2020 23:49:22
pybullet build time: Sep 20 2020 23:49:22

Add a minimum setup option that only contains the packages needed to run the most basic SMARTS scenarios.

The examples should be modularised with their own separate requirements so that their dependencies do not pollute the core of SMARTS.

Subtle bug w/ airlocking: if the shadowing actor returns done, it causes the vehicle to terminate. This should not be possible. Instead, only when the actor is actually in control can it influence vehicle termination. The solution likely involves updating the agent manager's observe method.

Add tests for removal in airlock and bubble.

• Update documentation
• Add a zoo specific Contributing file
  • Lay out the acceptance criteria
  • Reminder of their signing off on when contributing the policy
  • Describe how to pass the tests

Context:
The DAI competition has finished and some participants have trained models that did well during the competition. To adopt those models into our policy zoo, we need to provide guidance for them to submit their models and training code.

To reproduce:

1. Start envision server: scl envision start -s scenarios -p 8081
2. Replay: scl scenario replay -d data_replay/16031XXXXX

The data logs were put in the queue but never been sent to the Envision server.

Split from #17. Presently,

When a boid agent vehicle goes off road, a new sumo vehicle replace it and is stuck at the road side

We want the behavior to hand over control to SUMO and have SUMO attempt bringing the vehicle back on course. If the vehicle goes back into the bubble - not airlocking along the way - it should not get hijacked.

waypoints observation only provides waypoints in the vehicle's edge, but waypoints of the whole map or nearby edges should be given.

Potentially use the horizon parameter to define a search range.

Migrated from #64

• Set-up/find a Docker host for Docker image
• Check legal issues with hosting on Docker Hub
• Organise hand over maintenance responsibility
• Update README to indicate where to find the Docker image
• Update CI to use Docker image

After building a scenario with agents come from social agent zoo, twistd is still running.

https://github.com/huawei-noah/SMARTS/blob/master/cli/studio.py#L49

proc = subprocess.Popen("twistd", shell=True)
proc.terminate()

In this case, 2 processes will be spawned, one is the shell process and the other is the real twistd process (the shell's child process).

When terminate is called, only the shell process gets terminated.

Take into account waypoint direction when selecting nearest waypoints. The purpose of this is to help with navigating intersections.

See GL:!740 or ask for more context

Another proposed "elegant" solution: https://stackoverflow.com/a/21995928/2783780

One issue is that making left turns when taking the approach of using the least deviating waypoints as the course will cause the vehicle to drive into the outermost lane from the innermost lane because of the lower deviation in direction.

Easy to reproduce in all simulations view when running multi instances.

Update and cull dependencies:

• Update Torch
• Lock in black
• Cull requirements.txt packages

@iman512003 Please clarify the approach to take with this.

The current pybullet physics engine use takes up a bit too much of the simulation step between collision processing and physics simulation step.

Current plan: Move to a 2D physics model to improve performance

AgentManager, SMARTS, BubbleManager, and VehicleIndex each maintain some state duplication. This has led to bugs and reduced dev velocity because,

1. The state can easily get out of sync with each other
2. They each have their own vehicle lifecycle handling which leads to brittleness

The worst culprit is BubbleManager vs SMARTS.

Some notes from past discussions,

• Vehicles are being tracked, but those vehicles could "disappear", "done", etc.
  • BubbleManager does "roundabout" checks to determine these clauses (individually) and clean-itself up (i.e. remove cursors)
  • Vehicles spawning in the middle a bubble has special handling
    • Potential Solution: airlock based on time (e.g. airlock for N seconds) vs space
      • Airlock region is as large as the bubble, but we should airlock based on time instead
  • Becomes increasingly trickier w/ Boids
• Solutions
  1. Event subscription for underlaying vehicle state changes
     1. Event / "notification hub"
     2. Or could use event lifecycle
     3. Some challenges w/ this...
  2. Come back to this in the future: Remove stored state, and reference single source of truth (i.e. VehicleIndex)
     1. How to achieve that if we continue to need "Cursors"
        1. State machine
     2. May be able to get some inspiration from Drake use of global state
     3. step(...) interfaces would take a current state, and next state

Scenario loading should cache some of the re-usuable information shared between different scenarios.

This mainly includes:

• Waypoints
• Map geometry

Context: #43

A bug reported due to the bullet plane doesn't cover the map because our ground is a fixed size and location. We hardcoded the size 1M * 1M but not still big enough. Ideally, we should either:

• Provide an infinite plane if possible (Maybe btStaticPlaneShape is what we are looking for, but wasn't exposed in pybullet @iman512003)
• Update the origin and size of the plane during runtime

It's been failing for a long time and had been commented out after open sourcing.

Test to see if the correct waypoints are being given to observations

Follow-up to GL (Add tests for boid support)

We should write tests for these cases.

Context:

Nick: Current Error Message I had:

Traceback (most recent call last):
  File "/home/kyber/Desktop/SMARTS/smarts/core/smarts.py", line 133, in step
    return self._step(agent_actions)
  File "/home/kyber/Desktop/SMARTS/smarts/core/smarts.py", line 202, in _step
    self._bubble_manager.step(self)
  File "/home/kyber/Desktop/SMARTS/smarts/core/bubble_manager.py", line 191, in step
    state_change = self.step_bubble_state(social_vehicles, social_agent_vehicles)
  File "/home/kyber/Desktop/SMARTS/smarts/core/bubble_manager.py", line 236, in step_bubble_state
    if not cursor.update():
  File "/home/kyber/Desktop/SMARTS/smarts/core/bubble_manager.py", line 113, in update
    pos = Point(self.vehicle.position)
  File "/home/kyber/Desktop/SMARTS/smarts/core/vehicle.py", line 215, in position
    pos, _ = self._chassis.pose.as_panda3d()
  File "/home/kyber/Desktop/SMARTS/smarts/core/chassis.py", line 357, in pose
    pos, orn = self._client.getBasePositionAndOrientation(self._bullet_id)
TypeError: an integer is required (got type NoneType)

If running smarts gives you,

core/smarts.py", line 284, in setup
    provider_state = self._setup_providers(self._scenario)
  File "/Users/julian/src/SMARTS/smarts/core/smarts.py", line 550, in _setup_providers
    provider_state.merge(provider.setup(scenario))
  File "/Users/julian/src/SMARTS/smarts/core/sumo_traffic_simulation.py", line 199, in setup
    self._initialize_traci_conn()
  File "/Users/julian/src/SMARTS/smarts/core/sumo_traffic_simulation.py", line 123, in _initialize_traci_conn
    waitBetweenRetries=0.05,
  File "/usr/local/opt/sumo/share/sumo/tools/traci/main.py", line 121, in connect
    raise TraCIException("TraCI server already finished")
traci.exceptions.TraCIException: TraCI server already finished

or running netedit gives you something like,

[1]    1757 segmentation fault  netedit

and you're on macOS Catalina or newer then follow these steps to fix it eclipse-sumo/sumo#6242 (comment).

• Envision Readme: animation, data replay.
• Animation in docs/readme.rst
• Remove TOC in docs/CONTRIBUTING.md
• Starter kit in docs/readme.rst

The resulting recorded video from envision is not seek-able when using most video players. This makes the video much less useful.

RecordRTC documentation would suggest that getSeekableBlob would fix this issue: https://github.com/muaz-khan/RecordRTC/blob/master/README.md#global-apis

After initializing TraCI connection, SMARTS updates a few params by using traci_conn.load(load_params), which causes connecting SUMO twice. This will cause some overhead costs especially when running a large map.

Present problems of Envision rendering code:

• hard to read and debug

• current state management is not scalable and prone to error

Proposed solution:

• modularize indicators and meshes, replace setState with useRef

As a proof of concept, we can try few components first in the following orders:

• camera

• vehicle models

• road meshes

The rest of the components are:

• bubbles

• missions

• driven path

• waypoints

Done occuring during an environment reset causes AGENT_ID to not be in the observations after the next step.

import logging

import gym

from smarts.core.agent_interface import AgentInterface, AgentType
from smarts.core.agent import AgentSpec, Agent
from smarts.core.sensors import Observation

from examples import default_argument_parser


logging.basicConfig(level=logging.INFO)

AGENT_ID = "Agent-007"


class KeepLaneAgent(Agent):
    def act(self, obs: Observation):
        return "keep_lane"


def main(scenarios, max_episode_steps=None):
    agent_spec = AgentSpec(
        interface=AgentInterface.from_type(
            AgentType.Laner, max_episode_steps=max_episode_steps
        ),
        agent_builder=KeepLaneAgent,
    )

    env = gym.make(
        "smarts.env:hiway-v0",
        scenarios=scenarios,
        agent_specs={AGENT_ID: agent_spec},
        headless=False,
        visdom=False,
    )

    agent = agent_spec.build_agent()

    #done occurs here under the hood
    observations = env.reset()
    agent_obs = observations[AGENT_ID]
    agent_action = agent.act(agent_obs)

    # This works but should probably warn or assert because the agent no longer exists
    observations, rewards, dones, infos = env.step({AGENT_ID: agent_action})

    # This passes because done happened in `reset()` and the `env.step()` is no longer concerned with the agent.
    assert not dones["__all__"]

    # This crashes because the agent is done
    agent_obs = observations[AGENT_ID]

    env.close()


if __name__ == "__main__":
    parser = default_argument_parser("single-agent-example")
    args = parser.parse_args()

    main(
        scenarios=args.scenarios,
        max_episode_steps=0,
    )

The easiest way to reproduce this bug is to set the AgentInterface.max_episode_steps to 0. This will cause the agent to be done on the first step. However, there may be cases where the agent does terminate on the first step. We should find that out intelligently rather than crashing.

AC:

• SMARTS.reset() and env.reset() communicates agent dones
• SMARTS gives a warning when an agent is done on its first step.
• SMARTS provides a warning when trying to give an action for a done agent.

EDIT: example updated.

Configuration of smarts.env.hiway_env.HiWayEnv should provide the option for ordered and pseudo-randomised scenario execution.

The current implementation uses an iterator from Scenario.scenario_variations() that scrambles the scenario order. What is needed here is an additional option to HiWayEnv that will allow to specify that the execution order of scenarios should be the same as provided in the __init__().

The current signature of smarts.env.hiway_env.HiWayEnv:
https://github.com/huawei-noah/SMARTS/blob/master/smarts/env/hiway_env.py#L68-L84

The scenario iterator:
https://github.com/huawei-noah/SMARTS/blob/master/smarts/env/hiway_env.py#L91-L93

Scenario service characterized by social agent interaction.

1. Design the service, in terms of functionalities and interfaces. Once we have the design, then next is the architecture, think about what are we going to put behind the service, can we draw a solid line between the service and SMARTS, etc.
2. Based on the design, we can ship a workable prototype in 0.4.3.
3. Collect feedbacks from ChenDong's team and keep polishing the service.
4. Warp thing in the container.

Actions:

• Dockerfile @Adaickalavan
• Test building image and run the service inside a container @JingfeiPeng

There are multiple sub-issues here. Just porting this over to GH and keeping original intent.

• OGM and drivable area grid map seem not to be sent to boid agent even if registered in AgentInterface (fixed in #22)
• When a vehicle leaves a bubble, a new boid agent is created ("causing inference model to be instantiated again" ~ @Nickniyihan) (likely not an issue from discussion)
• When a boid agent vehicle goes off road, a new sumo vehicle replace it and is stuck at the road side: #23
• Boid agent should be created upon vehicle entering, and then sit at top of the bubble and always be there no matter boid agent vehicle traffic: #25

When a collision happens amongst boids we get the following exception,

Traceback (most recent call last):
  File "/home/kyber/Desktop/SMARTS/smarts/core/smarts.py", line 133, in step
    return self._step(agent_actions)
  File "/home/kyber/Desktop/SMARTS/smarts/core/smarts.py", line 202, in _step
    self._bubble_manager.step(self)
  File "/home/kyber/Desktop/SMARTS/smarts/core/bubble_manager.py", line 191, in step
    state_change = self.step_bubble_state(social_vehicles, social_agent_vehicles)
  File "/home/kyber/Desktop/SMARTS/smarts/core/bubble_manager.py", line 236, in step_bubble_state
    if not cursor.update():
  File "/home/kyber/Desktop/SMARTS/smarts/core/bubble_manager.py", line 113, in update
    pos = Point(self.vehicle.position)
  File "/home/kyber/Desktop/SMARTS/smarts/core/vehicle.py", line 215, in position
    pos, _ = self._chassis.pose.as_panda3d()
  File "/home/kyber/Desktop/SMARTS/smarts/core/chassis.py", line 357, in pose
    pos, orn = self._client.getBasePositionAndOrientation(self._bullet_id)
TypeError: an integer is required (got type NoneType)

This common behavior needs to be correctly handled.

When an agent vehicle is spawned in airlock, it doesn't get ogm/drivable area information, after a few steps it does. Related to the delay issue @aurorahcx

A few months ago, we made a change that forces the reinitialization of TRACI and the SUMO process to prevent the simulation from freezing. This change causes overhead costs, especially when switching between large maps.

A better way to decide whether to restart the SUMO process or not is based on the map. Ideally, we restart SUMO only if the map changes, otherwise, just reset everything.

There seems to be a Panda3D bug where the zBuffer ordering of vehicles to plane is random.

For now, a few assertions had been commented out to get it passed (see #14 & #31). Need to revisit the issue after open-sourcing.

Issues right now:

1. waypoints observation only provides waypoints in the vehicle's edge, but waypoints of the whole map should be given. (maybe something to do with RoadWaypoints.horizon)
2. Waypoints sequence should be aligned with lane direction. It is tricky when the waypoints are turning, which means heading change. (Still testing)
3. When the agent is spawned in airlock, it doesn't get ogm/drivable area information, but when it's inside bubble it does. Although this doesn't impact the model decision in the bubble, the problem should be fixed.

The process hangs while running the evaluation.

• Test trap successfully grabs vehicle
• Test trap emits vehicle with speed when it fails to capture a social vehicle
• Test trap does not capture pre-existing Agent vehicles
• Reactivation functionality removed

When SMARTS integrate with other simulators, other simulators might need to register/create vehicles upfront.

The context is that when SMARTS integrate with another simulator, we need to supply a *.rou.xml to SUMO to register a vehicle type and a route with a specific id so that simulator can create a vehicle with the type.

A *.rou.xml looks like:

<routes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://sumo.dlr.de/xsd/routes_file.xsd">
	<vType accel="0.8" color="0,0,1" decel="4.5" guiShape="passenger" height="1.5" id="IntelligentVehicle" length="5" maxSpeed="3" minGap="2.5" sigma="0.5" width="2"/>
	<route edges="A2-B2" id="ego_rou"/>
</routes>

This file is used to specify the vehicle type and route. No vehicle is created.

A travelling bubble is attached/pinned to a vehicle while maintaining its relative pose.

• Trap management #63
• Providers #120

Change over to python bindings in the OpEn agent

• Linux bindings: #69

We want 5 social agents all on different machines contributing towards one simulation.

core

• Seamless hijack of vehicles going through bubbles.
• A single agent in the simulation from a remote machine
• 5 agents from remote machines

prep work

• #88 Merge Agent and Policy
• #89 RemoteAgent to use async def act(obs) instead of send_obs(), recv_act()
• #53 Pool social agents by agent_specs.

networking

• [ ]

Find and weed out the following issues with SUMO vehicles disregarding the agent vehicles:

Promise to investigate

The metrics that need to be considered:

• Jerkness
• Acceleration
• Sliding or not
• Change of angle

cvxopt and quadprog use GPLV2/3 licenses, which should be fine per our discussions.

The main issue with using these libraries in critical sections of SMARTS is the GPL requirements it places on users of SMARTS to also keep their projects open-source permissive.

Presently only target pose is supported. We'll want to support both of these controller types.