Till sidans topp

Sidansvarig: Webbredaktion
Sidan uppdaterades: 2012-09-11 15:12

Tipsa en vän
Utskriftsversion

MoVEMo: A Structured Appr… - Göteborgs universitet Till startsida
Webbkarta
Till innehåll Läs mer om hur kakor används på gu.se

MoVEMo: A Structured Approach for Engineering Reward Functions

Paper i proceeding
Författare Mallozzi Piergiuseppe
Raul Pardo
Vincent Duplessis
Patrizio Pelliccione
Gerardo Schneider
Publicerad i 2nd IEEE International Conference on Robotic Computing (IRC)
ISBN 978-1-5386-4651-9
Förlag IEEE
Förlagsort New York
Publiceringsår 2018
Publicerad vid Institutionen för data- och informationsteknik (GU)
Språk en
Länkar doi.org/10.1109/IRC.2018.00053
Ämneskategorier Datalogi, Programvaruteknik

Sammanfattning

Reinforcement learning (RL) is a machine learning technique that has been increasingly used in robotic systems. In reinforcement learning, instead of manually pre-program what action to take at each step, we convey the goal a software agent in terms of reward functions. The agent tries different actions in order to maximize a numerical value, i. e. the reward. A misspecified reward function can cause problems such as reward hacking, where the agent finds out ways that maximize the reward without achieving the intended goal. As RL agents become more general and autonomous, the design of reward functions that elicit the desired behaviour in the agent becomes more important and cumbersome. In this paper, we present a technique to formally express reward functions in a structured way; this stimulates a proper reward function design and as well enables the formal verification of it. We start by defining the reward function using state machines. In this way, we can statically check that the reward function satisfies certain properties, e. g., high-level requirements of the function to learn. Later we automatically generate a runtime monitor-which runs in parallel with the learning agent-that provides the rewards according to the definition of the state machine and based on the behaviour of the agent. We use the UPPAAL model checker to design the reward model and verify the TCTL properties that model high-level requirements of the reward function and LARVA to monitor and enforce the reward model to the RL agent at runtime.

Sidansvarig: Webbredaktion|Sidan uppdaterades: 2012-09-11
Dela:

På Göteborgs universitet använder vi kakor (cookies) för att webbplatsen ska fungera på ett bra sätt för dig. Genom att surfa vidare godkänner du att vi använder kakor.  Vad är kakor?