{"id":12,"date":"2019-04-30T20:08:06","date_gmt":"2019-04-30T20:08:06","guid":{"rendered":"http:\/\/groups.cs.umass.edu\/shlomo\/?page_id=12"},"modified":"2022-01-02T16:12:06","modified_gmt":"2022-01-02T16:12:06","slug":"research","status":"publish","type":"page","link":"https:\/\/groups.cs.umass.edu\/shlomo\/research\/","title":{"rendered":"Research"},"content":{"rendered":"

We study a wide range of problems in artificial intelligence, automated planning and learning, autonomous systems, reasoning under uncertainty, multi-agent systems, and resource-bounded reasoning. We are particularly interested in the implications of uncertainty and limited computational resources on the design of autonomous agents. In most practical settings, it is not feasible or desirable to find the optimal action, making it necessary to resort to some form of approximate reasoning. This raises a fundamental question: what does it mean for an agent to be \u201crational\u201d when it does not have enough knowledge or computational power to derive the best course of action? Our overall approach to this problem involves meta-level control mechanisms that reason explicitly about the cost of decision-making and can optimize the amount of deliberation (or “thinking”) an agent does before taking action. We have also developed new planning techniques for situations involving multiple decision makers operating in either collaborative or adversarial domains.<\/p>\n

\"\"<\/p>\n

Human Compatible AI<\/b><\/span><\/h3>\n
How can we design AI systems that are compatible with human needs: accountable, explainable, equitable, ethical, and mindful of human cognitive biases and shortcomings?<\/div>\n
Show Related Publications<\/a>
<\/a><\/form>

Miura, Shuwa; Zilberstein, Shlomo<\/p>

Observer-Aware Planning with Implicit and Explicit Communication<\/a> Conference<\/span> <\/p>

Proceedings of the The 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS), <\/span>Auckland, New Zealand, <\/span>2024<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:MZaamas24,
\r\ntitle = {Observer-Aware Planning with Implicit and Explicit Communication},
\r\nauthor = {Shuwa Miura and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/MZaamas24.pdf},
\r\nyear  = {2024},
\r\ndate = {2024-01-01},
\r\nbooktitle = {Proceedings of the The 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS)},
\r\naddress = {Auckland, New Zealand},
\r\nabstract = {This paper presents a computational model designed for planning both implicit and explicit communication of intentions, goals, and desires. Building upon previous research focused on implicit communication of intention via actions, our model seeks to strategically influence an observer\u2019s belief using both the agent\u2019s actions and explicit messages. We show that our proposed model can be considered to be a special case of general multi-agent problems with explicit communication under certain assumptions. Since the mental state of the observer depends on histories, computing a policy for the proposed model amounts to optimizing a non-Markovian objective, which we show to be intractable in the worst case. To mitigate this challenge, we propose a technique based on splitting domain and communication actions during planning. We conclude with experimental evaluations of the proposed approach that illustrate its effectiveness.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Mahmud, Saaduddin; Vazquez-Chanlatte, Marcell; Witwicki, Stefan; Zilberstein, Shlomo<\/p>

Explaining the Behavior of POMDP-based Agents Through the Impact of Counterfactual Information<\/a> Conference<\/span> <\/p>

Proceedings of the The 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS), <\/span>Auckland, New Zealand, <\/span>2024<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:MVWZaamas24,
\r\ntitle = {Explaining the Behavior of POMDP-based Agents Through the Impact of Counterfactual Information},
\r\nauthor = {Saaduddin Mahmud and Marcell Vazquez-Chanlatte and Stefan Witwicki and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/MVWZaamas24.pdf},
\r\nyear  = {2024},
\r\ndate = {2024-01-01},
\r\nbooktitle = {Proceedings of the The 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS)},
\r\naddress = {Auckland, New Zealand},
\r\nabstract = {In this work, we consider AI agents operating in Partially Observable Markov Decision Processes (POMDPs)\u2013a widely-used framework for sequential decision making with incomplete state information. Agents operating with partial information take actions not only to advance their underlying goals but also to seek information and reduce uncertainty. Despite rapid progress in explainable AI, research on separating information-driven vs. goal-driven behaviors remains sparse. To address this gap, we introduce a novel explanation generation framework called Sequential Information Probing (SIP), to investigate the direct impact of state information, or its absence, on agent behavior. To quantify the impact we also propose two metrics under this SIP framework called Value of Information (VoI) and Influence of Information (IoI). We then theoretically derive several properties of these metrics. Finally, we present several experiments, including a case study on an autonomous vehicle, that illustrate the efficacy of our method.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Choudhury, Moumita; Saisubramanian, Sandhya; Zhang, Hao; Zilberstein, Shlomo<\/p>

Minimizing Negative Side Effects in Cooperative Multi-Agent Systems Using Distributed Coordination<\/a> Conference<\/span> <\/p>

Proceedings of the The 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS), <\/span>Auckland, New Zealand, <\/span>2024<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:CSZZaamas24,
\r\ntitle = {Minimizing Negative Side Effects in Cooperative Multi-Agent Systems Using Distributed Coordination},
\r\nauthor = {Moumita Choudhury and Sandhya Saisubramanian and Hao Zhang and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/CSZZaamas24.pdf},
\r\nyear  = {2024},
\r\ndate = {2024-01-01},
\r\nbooktitle = {Proceedings of the The 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS)},
\r\naddress = {Auckland, New Zealand},
\r\nabstract = {Autonomous agents in real-world environments may encounter undesirable outcomes or negative side effects (NSEs) when working collaboratively alongside other agents. We frame the challenge of minimizing NSEs in a multi-agent setting as a lexicographic decentralized Markov decision process in which we assume independence of rewards and transitions with respect to the primary assigned tasks, but allowing negative side effects to create a form of dependence among the agents. We present a lexicographic Q-learning approach to mitigate the NSEs using human feedback models while maintaining near-optimality with respect to the assigned tasks\u2013up to some given slack. Our empirical evaluation across two domains demonstrates that our collaborative approach effectively mitigates NSEs, outperforming non-collaborative methods.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Choudhury, Moumita; Saisubramanian, Sandhya; Zhang, Hao; Zilberstein, Shlomo<\/p>

Minimizing Negative Side Effects in Cooperative Multi-Agent Systems Using Distributed Coordination<\/a> Conference<\/span> <\/p>

Proceedings of the The 37th International FLAIRS Conference, <\/span>Miramar Beach, Florida, <\/span>2024<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:CSZZflairs24,
\r\ntitle = {Minimizing Negative Side Effects in Cooperative Multi-Agent Systems Using Distributed Coordination},
\r\nauthor = {Moumita Choudhury and Sandhya Saisubramanian and Hao Zhang and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/CSZZflairs24.pdf},
\r\nyear  = {2024},
\r\ndate = {2024-01-01},
\r\nbooktitle = {Proceedings of the The 37th International FLAIRS Conference},
\r\naddress = {Miramar Beach, Florida},
\r\nabstract = {Autonomous agents operating in real-world environments frequently encounter undesirable outcomes or negative side effects (NSEs) when working collaboratively alongside other agents. Even when agents can execute their primary task optimally when operating in isolation, their training may not account for potential negative interactions that arise in the presence of other agents. We frame the challenge of minimizing NSEs as a lexicographic decentralized Markov decision process in which we assume independence of rewards and transitions with respect to the primary assigned tasks, but recognize that addressing negative side effects creates a form of dependence among the agents. We present a lexicographic Q-learning approach to mitigate the NSEs using human feedback models while maintaining near-optimality with respect to the assigned tasks\u2013up to some given slack. Our empirical evaluation across two domains demonstrates that our collaborative approach effectively mitigates NSEs, outperforming non-collaborative methods.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Mahmud, Saaduddin; Nashed, Samer B.; Goldman, Claudia V.; Zilberstein, Shlomo<\/p>

Estimating Causal Responsibility for Explaining Autonomous Behavior<\/a> Book Section<\/span> <\/p>

In: <\/span> Calvaresi, Davide (Ed.): International Workshop on Explainable and Transparent AI and Multi-Agent Systems (EXTRAAMAS), <\/span>pp. 78\u201394, <\/span>Springer, <\/span>2023<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@incollection{SZ:MNGZextraamas23,
\r\ntitle = {Estimating Causal Responsibility for Explaining Autonomous Behavior},
\r\nauthor = {Saaduddin Mahmud and Samer B. Nashed and Claudia V. Goldman and Shlomo Zilberstein},
\r\neditor = {Davide Calvaresi},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/MNGZextraamas23.pdf},
\r\ndoi = {10.1007\/978-3-031-40878-6},
\r\nyear  = {2023},
\r\ndate = {2023-01-01},
\r\nbooktitle = {International Workshop on Explainable and Transparent AI and Multi-Agent Systems (EXTRAAMAS)},
\r\npages = {78\u201394},
\r\npublisher = {Springer},
\r\nabstract = {There has been growing interest in causal explanations of stochastic, sequential decision-making systems. Structural causal models and causal reasoning offer several theoretical benefits when exact inference can be applied. Furthermore, users overwhelmingly prefer the resulting causal explanations over other state-of-the-art systems. In this work, we focus on one such method, MeanRESP, and its approximate versions that drastically reduce compute load and assign a responsibility score to each variable, which helps identify smaller sets of causes to be used as explanations. However, this method, and its approximate versions in particular, lack deeper theoretical analysis and broader empirical tests. To address these shortcomings, we provide three primary contributions. First, we offer several theoretical insights on the sample complexity and error rate of approximate MeanRESP. Second, we discuss several automated metrics for comparing explanations generated from approximate methods to those generated via exact methods. While we recognize the significance of user studies as the gold standard for evaluating explanations, our aim is to leverage the proposed metrics to systematically compare explanation-generation methods along important quantitative dimensions. Finally, we provide a more detailed discussion of MeanRESP and how its output under different definitions of responsibility compares to existing widely adopted methods that use Shapley values.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {incollection}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>
There has been growing interest in causal explanations of stochastic, sequential decision-making systems. Structural causal models and causal reasoning offer several theoretical benefits when exact inference can be applied. Furthermore, users overwhelmingly prefer the resulting causal explanations over other state-of-the-art systems. In this work, we focus on one such method, MeanRESP, and its approximate versions that drastically reduce compute load and assign a responsibility score to each variable, which helps identify smaller sets of causes to be used as explanations. However, this method, and its approximate versions in particular, lack deeper theoretical analysis and broader empirical tests. To address these shortcomings, we provide three primary contributions. First, we offer several theoretical insights on the sample complexity and error rate of approximate MeanRESP. Second, we discuss several automated metrics for comparing explanations generated from approximate methods to those generated via exact methods. While we recognize the significance of user studies as the gold standard for evaluating explanations, our aim is to leverage the proposed metrics to systematically compare explanation-generation methods along important quantitative dimensions. Finally, we provide a more detailed discussion of MeanRESP and how its output under different definitions of responsibility compares to existing widely adopted methods that use Shapley values.<\/div>
Close<\/a><\/p><\/div>

Saisubramanian, Sandhya; Zilberstein, Shlomo; Kamar, Ece<\/p>

Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems<\/a> Journal Article<\/span> <\/p>

In: <\/span>AI Magazine, <\/span>vol. 42, <\/span>no. 4, <\/span>pp. 62\u201371, <\/span>2022<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@article{SZ:SZKaimag22,
\r\ntitle = {Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems},
\r\nauthor = {Sandhya Saisubramanian and Shlomo Zilberstein and Ece Kamar},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/SZKaimag22.pdf},
\r\ndoi = {\u000e10.1609\/aaai.12028},
\r\nyear  = {2022},
\r\ndate = {2022-01-01},
\r\nurldate = {2022-01-01},
\r\njournal = {AI Magazine},
\r\nvolume = {42},
\r\nnumber = {4},
\r\npages = {62--71},
\r\nabstract = {Autonomous agents acting in the real-world often operate based on models that ignore certain aspects of the environment. The incompleteness of any given model \u2013 handcrafted or machine acquired \u2013 is inevitable due to practical limitations of any modeling technique for complex real-world settings. Due to the limited fidelity of its model, an agent\u2019s actions may have unexpected, undesirable consequences during execution. Learning to recognize and avoid such negative side effects (NSEs) of an agent\u2019s actions is critical to improve the safety and reliability of autonomous systems. Mitigating NSEs is an emerging research topic that is attracting increased attention due to the rapid growth in the deployment of AI systems and their broad societal impacts. This article provides a comprehensive overview of different forms of NSEs and the recent research efforts to address them. We identify key characteristics of NSEs, highlight the challenges in avoiding NSEs, and discuss recently developed approaches, contrasting their benefits and limitations. The article concludes with a discussion of open questions and suggestions for future research directions.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {article}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Saisubramanian, Sandhya; Zilberstein, Shlomo; Kamar, Ece<\/p>

Avoiding Negative Side Effects of Autonomous Systems in the Open World<\/a> Journal Article<\/span> <\/p>

In: <\/span>Journal of Artificial Intelligence Research (JAIR), <\/span>vol. 74, <\/span>pp. 143\u2013177, <\/span>2022<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@article{SZ:SZKjair22,
\r\ntitle = {Avoiding Negative Side Effects of Autonomous Systems in the Open World},
\r\nauthor = {Sandhya Saisubramanian and Shlomo Zilberstein and Ece Kamar},
\r\nurl = {https:\/\/www.jair.org\/index.php\/jair\/article\/view\/13581\/26799},
\r\ndoi = {10.1613\/jair.1.13581},
\r\nyear  = {2022},
\r\ndate = {2022-01-01},
\r\nurldate = {2022-01-01},
\r\njournal = {Journal of Artificial Intelligence Research (JAIR)},
\r\nvolume = {74},
\r\npages = {143--177},
\r\nabstract = {Autonomous systems that operate in the open world often use incomplete models of their environment. Model incompleteness is inevitable due to the practical limitations in precise model specification and data collection about open-world environments. Due to the limited fidelity of the model, agent actions may produce negative side effects (NSEs) when deployed. Negative side effects are undesirable, unmodeled effects of agent actions on the environment. NSEs are inherently challenging to identify at design time and may affect the reliability, usability and safety of the system. We present two complementary approaches to mitigate the NSE via: (1) learning from feedback, and (2) environment shaping. The solution approaches target settings with different assumptions and agent responsibilities. In learning from feedback, the agent learns a penalty function associated with a NSE. We investigate the efficiency of different feedback mechanisms, including human feedback and autonomous exploration. The problem is formulated as a multi-objective Markov decision process such that optimizing the agent\u2019s assigned task is prioritized over mitigating NSE. A slack parameter denotes the maximum allowed deviation from the optimal expected reward for the agent\u2019s task in order to mitigate NSE. In environment shaping, we examine how a human can assist an agent, beyond providing feedback, and utilize their broader scope of knowledge to mitigate the impacts of NSE. We formulate the problem as a human-agent collaboration with decoupled objectives. The agent optimizes its assigned task and may produce NSE during its operation. The human assists the agent by performing modest reconfigurations of the environment so as to mitigate the impacts of NSE, without affecting the agent\u2019s ability to complete its assigned task. We present an algorithm for shaping and analyze its properties. Empirical evaluations demonstrate the trade-offs in the performance of different approaches in mitigating NSE in different settings.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {article}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>
Autonomous systems that operate in the open world often use incomplete models of their environment. Model incompleteness is inevitable due to the practical limitations in precise model specification and data collection about open-world environments. Due to the limited fidelity of the model, agent actions may produce negative side effects (NSEs) when deployed. Negative side effects are undesirable, unmodeled effects of agent actions on the environment. NSEs are inherently challenging to identify at design time and may affect the reliability, usability and safety of the system. We present two complementary approaches to mitigate the NSE via: (1) learning from feedback, and (2) environment shaping. The solution approaches target settings with different assumptions and agent responsibilities. In learning from feedback, the agent learns a penalty function associated with a NSE. We investigate the efficiency of different feedback mechanisms, including human feedback and autonomous exploration. The problem is formulated as a multi-objective Markov decision process such that optimizing the agent\u2019s assigned task is prioritized over mitigating NSE. A slack parameter denotes the maximum allowed deviation from the optimal expected reward for the agent\u2019s task in order to mitigate NSE. In environment shaping, we examine how a human can assist an agent, beyond providing feedback, and utilize their broader scope of knowledge to mitigate the impacts of NSE. We formulate the problem as a human-agent collaboration with decoupled objectives. The agent optimizes its assigned task and may produce NSE during its operation. The human assists the agent by performing modest reconfigurations of the environment so as to mitigate the impacts of NSE, without affecting the agent\u2019s ability to complete its assigned task. We present an algorithm for shaping and analyze its properties. Empirical evaluations demonstrate the trade-offs in the performance of different approaches in mitigating NSE in different settings.<\/div>
Close<\/a><\/p><\/div>

Miura, Shuwa; Wray, Kyle Hollins; Zilberstein, Shlomo<\/p>

Heuristic Search for SSPs with Lexicographic Preferences over Multiple Costs<\/a> Conference<\/span> <\/p>

Proceedings of the 15th Annual Symposium on Combinatorial Search (SOCS), <\/span>Vienna, Austria, <\/span>2022<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:MWZsocs22,
\r\ntitle = {Heuristic Search for SSPs with Lexicographic Preferences over Multiple Costs},
\r\nauthor = {Shuwa Miura and Kyle Hollins Wray and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/MWZsocs22.pdf},
\r\nyear  = {2022},
\r\ndate = {2022-01-01},
\r\nbooktitle = {Proceedings of the 15th Annual Symposium on Combinatorial Search (SOCS)},
\r\naddress = {Vienna, Austria},
\r\nabstract = {Real-world decision problems often involve multiple competing objectives. The Stochastic Shortest Path (SSP) with lexicographic preferences over multiple costs offers an expressive formulation for many practical problems. However, the existing solution methods either lack optimality guarantees or require costly computations over the entire state space. We propose the first heuristic search algorithm for this problem, based on the heuristic algorithm for Constrained SSPs. Our experiments show that our heuristic search algorithm can compute optimal policies while avoiding a large portion of the state space. We also analyze the theoretical properties of the problem, establishing the conditions under which SSPs with lexicographic preferences have a proper optimal policy.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Svegliato, Justin; Nashed, Samer B; Zilberstein, Shlomo<\/p>

Ethically Compliant Sequential Decision Making<\/a> Conference<\/span> <\/p>

Proceedings of the 35th Conference on Artificial Intelligence (AAAI), <\/span>2021<\/span>, (Distinguished Paper Award)<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:SNZaaai21,
\r\ntitle = {Ethically Compliant Sequential Decision Making},
\r\nauthor = {Justin Svegliato and Samer B Nashed and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/SNZaaai21.pdf},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\nbooktitle = {Proceedings of the 35th Conference on Artificial Intelligence (AAAI)},
\r\npages = {11657--11665},
\r\nabstract = {Enabling autonomous systems to comply with an ethical theory is critical given their accelerating deployment in domains that impact society. While many ethical theories have been studied extensively in moral philosophy, they are still challenging to implement by developers who build autonomous systems. This paper proposes a novel approach for building ethically compliant autonomous systems that optimize completing a task while following an ethical framework. First, we introduce a definition of an ethically compliant autonomous system and its properties. Next, we offer a range of ethical frameworks for divine command theory, prima facie duties, and virtue ethics. Finally, we demonstrate the accuracy and usability of our approach in a set of autonomous driving simulations and a user study of planning and robotics experts.},
\r\nnote = {Distinguished Paper Award},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Miura, Shuwa; Cohen, Andrew L; Zilberstein, Shlomo<\/p>

Maximizing Legibility in Stochastic Environments<\/a> Conference<\/span> <\/p>

Proceedings of the 30th IEEE International Conference on Robot & Human Interactive Communication, (RO-MAN), <\/span>Vancouver, BC, Canada, <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:MCZroman21,
\r\ntitle = {Maximizing Legibility in Stochastic Environments},
\r\nauthor = {Shuwa Miura and Andrew L Cohen and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/MCZroman21.pdf},
\r\ndoi = {10.1109\/RO-MAN50785.2021.9515318},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\nbooktitle = {Proceedings of the 30th IEEE International Conference on Robot & Human Interactive Communication, (RO-MAN)},
\r\npages = {1053--1059},
\r\naddress = {Vancouver, BC, Canada},
\r\nabstract = {Making an agent's intentions clear from its observed behavior is crucial for seamless human-agent interaction and for increased transparency and trust in AI systems. Existing methods that address this challenge and maximize legibility of behaviors are limited to deterministic domains. We develop a technique for maximizing legibility in stochastic environments and illustrate that using legibility as an objective improves interpretability of agent behavior in several scenarios. We provide initial empirical evidence that human subjects can better interpret legible behavior.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Miura, Shuwa; Zilberstein, Shlomo<\/p>

A Unifying Framework for Observer-Aware Planning and its Complexity<\/a> Conference<\/span> <\/p>

Proceedings of the 37th Conference on Uncertainty in Artificial Intelligence (UAI), <\/span>Virtual Event, <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:MZuai21,
\r\ntitle = {A Unifying Framework for Observer-Aware Planning and its Complexity},
\r\nauthor = {Shuwa Miura and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/MZuai21.pdf},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\nbooktitle = {Proceedings of the 37th Conference on Uncertainty in Artificial Intelligence (UAI)},
\r\npages = {610--620},
\r\naddress = {Virtual Event},
\r\nabstract = {Being aware of observers and the inferences they make about an agent's behavior is crucial for successful multi-agent interaction. Existing works on observer-aware planning use different assumptions and techniques to produce observer-aware behaviors. We argue that observer-aware planning, in its most general form, can be modeled as an Interactive POMDP (I-POMDP), which requires complex modeling and is hard to solve. Hence, we introduce a less complex framework for producing observer-aware behaviors called Observer-Aware MDP (OAMDP) and analyze its relationship to I-POMDP. We establish the complexity of OAMDPs and show that they can improve interpretability of agent behaviors in several scenarios.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Rabiee, Sadegh; Basich, Connor; Wray, Kyle Hollins; Zilberstein, Shlomo; Biswas, Joydeep<\/p>

Competence-Aware Path Planning via Introspective Perception<\/a> Journal Article<\/span> <\/p>

In: <\/span>CoRR, <\/span>vol. abs\/2109.13974, <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@article{SZ:SZarXiv21c,
\r\ntitle = {Competence-Aware Path Planning via Introspective Perception},
\r\nauthor = {Sadegh Rabiee and Connor Basich and Kyle Hollins Wray and Shlomo Zilberstein and Joydeep Biswas},
\r\nurl = {https:\/\/arxiv.org\/abs\/2109.13974},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\njournal = {CoRR},
\r\nvolume = {abs\/2109.13974},
\r\nabstract = {Robots deployed in the real world over extended periods of time need to reason about unexpected failures, learn to predict them, and to proactively take actions to avoid future failures. Existing approaches for competence-aware planning are either model-based, requiring explicit enumeration of known failure modes, or purely statistical, using state- and location-specific failure statistics to infer competence. We instead propose a structured model-free approach to competence-aware planning by reasoning about plan execution failures due to errors in perception, without requiring a-priori enumeration of failure modes or requiring location-specific failure statistics. We introduce competence-aware path planning via introspective perception (CPIP), a Bayesian framework to iteratively learn and exploit task-level competence in novel deployment environments. CPIP factorizes the competence-aware planning problem into two components. First, perception errors are learned in a model-free and location-agnostic setting via introspective perception prior to deployment in novel environments. Second, during actual deployments, the prediction of task-level failures is learned in a context-aware setting. Experiments in a simulation show that the proposed CPIP approach outperforms the frequentist baseline in multiple mobile robot tasks, and is further validated via real robot experiments in an environment with perceptually challenging obstacles and terrain.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {article}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>
Robots deployed in the real world over extended periods of time need to reason about unexpected failures, learn to predict them, and to proactively take actions to avoid future failures. Existing approaches for competence-aware planning are either model-based, requiring explicit enumeration of known failure modes, or purely statistical, using state- and location-specific failure statistics to infer competence. We instead propose a structured model-free approach to competence-aware planning by reasoning about plan execution failures due to errors in perception, without requiring a-priori enumeration of failure modes or requiring location-specific failure statistics. We introduce competence-aware path planning via introspective perception (CPIP), a Bayesian framework to iteratively learn and exploit task-level competence in novel deployment environments. CPIP factorizes the competence-aware planning problem into two components. First, perception errors are learned in a model-free and location-agnostic setting via introspective perception prior to deployment in novel environments. Second, during actual deployments, the prediction of task-level failures is learned in a context-aware setting. Experiments in a simulation show that the proposed CPIP approach outperforms the frequentist baseline in multiple mobile robot tasks, and is further validated via real robot experiments in an environment with perceptually challenging obstacles and terrain.<\/div>
Close<\/a><\/p><\/div>

Nashed, Samer B; Svegliato, Justin; Zilberstein, Shlomo<\/p>

Ethically Compliant Planning within Moral Communities<\/a> Conference<\/span> <\/p>

Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society (AIES), <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:NSZaies21,
\r\ntitle = {Ethically Compliant Planning within Moral Communities},
\r\nauthor = {Samer B Nashed and Justin Svegliato and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/NSZaies21.pdf},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\nbooktitle = {Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society (AIES)},
\r\nabstract = {Ethically compliant autonomous systems (ECAS) are the state-of- the-art for solving sequential decision-making problems under un- certainty while respecting constraints that encode ethical considerations. This paper defines a novel concept in the context of ECAS that is from moral philosophy, the moral community, which leads to a nuanced taxonomy of explicit ethical agents. We then propose new ethical frameworks that extend the applicability of ECAS to domains where a moral community is required. Next, we provide a formal analysis of the proposed ethical frameworks and conduct experiments that illustrate their differences. Finally, we discuss the implications of explicit moral communities that could shape research on standards and guidelines for ethical agents in order to better understand and predict common errors in their design and communicate their capabilities.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Galhotra, Sainyam; Saisubramanian, Sandhya; Zilberstein, Shlomo<\/p>

Learning to Generate Fair Clusters from Demonstrations<\/a> Conference<\/span> <\/p>

Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society (AIES), <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:GSZaies21,
\r\ntitle = {Learning to Generate Fair Clusters from Demonstrations},
\r\nauthor = {Sainyam Galhotra and Sandhya Saisubramanian and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/GSZaies21.pdf},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\nbooktitle = {Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society (AIES)},
\r\nabstract = {Fair clustering is the process of grouping similar entities together, while satisfying a mathematically well-defined fairness metric as a constraint. Due to the practical challenges in precise model specification, the prescribed fairness constraints are often incomplete and act as proxies to the intended fairness requirement. Clustering with proxies may lead to biased outcomes when the system is deployed. We examine how to identify the intended fairness constraint for a problem based on limited demonstrations from an expert. Each demonstration is a clustering over a subset of the data. We present an algorithm to identify the fairness metric from demonstrations and generate clusters using existing off-the-shelf clustering techniques, and analyze its theoretical properties. To extend our approach to novel fairness metrics for which clustering algorithms do not currently exist, we present a greedy method for clustering. Additionally, we investigate how to generate interpretable solutions using our approach. Empirical evaluation on three real-world datasets demonstrates the effectiveness of our approach in quickly identifying the underlying fairness and interpretability constraints, which are then used to generate fair and interpretable clusters.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Galhotra, Sainyam; Saisubramanian, Sandhya; Zilberstein, Shlomo<\/p>

Learning to Generate Fair Clusters from Demonstrations<\/a> Journal Article<\/span> <\/p>

In: <\/span>CoRR, <\/span>vol. abs\/2102.03977, <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@article{SZ:GSZarXiv21a,
\r\ntitle = {Learning to Generate Fair Clusters from Demonstrations},
\r\nauthor = {Sainyam Galhotra and Sandhya Saisubramanian and Shlomo Zilberstein},
\r\nurl = {https:\/\/arxiv.org\/abs\/2102.03977},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\njournal = {CoRR},
\r\nvolume = {abs\/2102.03977},
\r\nabstract = {Fair clustering is the process of grouping similar entities together, while satisfying a mathematically well-defined fairness metric as a constraint. Due to the practical challenges in precise model specification, the prescribed fairness constraints are often incomplete and act as proxies to the intended fairness requirement, leading to biased outcomes when the system is deployed. We examine how to identify the intended fairness constraint for a problem based on limited demonstrations from an expert. Each demonstration is a clustering over a subset of the data. We present an algorithm to identify the fairness metric from demonstrations and generate clusters using existing off-the-shelf clustering techniques, and analyze its theoretical properties. To extend our approach to novel fairness metrics for which clustering algorithms do not currently exist, we present a greedy method for clustering. Additionally, we investigate how to generate interpretable solutions using our approach. Empirical evaluation on three real-world datasets demonstrates the effectiveness of our approach in quickly identifying the underlying fairness and interpretability constraints, which are then used to generate fair and interpretable clusters.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {article}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Saisubramanian, Sandhya; Zilberstein, Shlomo<\/p>

Mitigating Negative Side Effects via Environment Shaping<\/a> Journal Article<\/span> <\/p>

In: <\/span>CoRR, <\/span>vol. abs\/2102.07017, <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@article{SZ:SZarXiv21b,
\r\ntitle = {Mitigating Negative Side Effects via Environment Shaping},
\r\nauthor = {Sandhya Saisubramanian and Shlomo Zilberstein},
\r\nurl = {https:\/\/arxiv.org\/abs\/2102.07017},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\njournal = {CoRR},
\r\nvolume = {abs\/2102.07017},
\r\nabstract = {Agents operating in unstructured environments often produce negative side effects (NSE), which are difficult to identify at design time. While the agent can learn to mitigate the side effects from human feedback, such feedback is often expensive and the rate of learning is sensitive to the agent's state representation. We examine how humans can assist an agent, beyond providing feedback, and exploit their broader scope of knowledge to mitigate the impacts of NSE. We formulate this problem as a human-agent team with decoupled objectives. The agent optimizes its assigned task, during which its actions may produce NSE. The human shapes the environment through minor reconfiguration actions so as to mitigate the impacts of the agent's side effects, without affecting the agent's ability to complete its assigned task. We present an algorithm to solve this problem and analyze its theoretical properties. Through experiments with human subjects, we assess the willingness of users to perform minor environment modifications to mitigate the impacts of NSE. Empirical evaluation of our approach shows that the proposed framework can successfully mitigate NSE, without affecting the agent's ability to complete its assigned task.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {article}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Saisubramanian, Sandhya; Zilberstein, Shlomo<\/p>

Mitigating Negative Side Effects via Environment Shaping (Extended Abstract)<\/a> Conference<\/span> <\/p>

Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:SZaamas21,
\r\ntitle = {Mitigating Negative Side Effects via Environment Shaping (Extended Abstract)},
\r\nauthor = {Sandhya Saisubramanian and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/SZaamas21.pdf},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\nbooktitle = {Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS)},
\r\nabstract = {Agents operating in the open world often produce negative side effects (NSE), which are difficult to identify at design time. We examine how a human can assist an agent, beyond providing feedback, and exploit their broader scope of knowledge to mitigate the impacts of NSE. We formulate this problem as a human-agent team with decoupled objectives. The agent optimizes its assigned task, during which its actions may produce NSE. The human shapes the environment through minor reconfiguration actions so as to mitigate the impacts of agent's side effects, without significantly degrading agent performance. We present an algorithm to solve this problem. Empirical evaluation shows that the proposed framework can successfully mitigate NSE, without affecting the agent\u2019s ability to complete its assigned task.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Saisubramanian, Sandhya; Roberts, Shannon C; Zilberstein, Shlomo<\/p>

Understanding User Attitudes Towards Negative Side Effects of AI Systems<\/a> Conference<\/span> <\/p>

CHI Conference on Human Factors in Computing Systems, Late-Breaking Work, <\/span>2021<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:SRZchi21,
\r\ntitle = {Understanding User Attitudes Towards Negative Side Effects of AI Systems},
\r\nauthor = {Sandhya Saisubramanian and Shannon C Roberts and Shlomo Zilberstein},
\r\nurl = {http:\/\/rbr.cs.umass.edu\/shlomo\/papers\/SRZchi21.pdf},
\r\nyear  = {2021},
\r\ndate = {2021-01-01},
\r\nbooktitle = {CHI Conference on Human Factors in Computing Systems, Late-Breaking Work},
\r\npages = {368:1--368:6},
\r\nabstract = {Artificial Intelligence (AI) systems deployed in the open world may produce negative side effects\u2014which are unanticipated, undesirable outcomes that occur in addition to the intended outcomes of the system\u2019s actions. These negative side effects affect users directly or indirectly, by violating their preferences or altering their environment in an undesirable, potentially harmful, manner. While the existing literature has started to explore techniques to overcome the impacts of negative side effects in deployed systems, there has been no prior efforts to determine how users perceive and respond to negative side effects. We surveyed 183 participants to develop an understanding of user attitudes towards side effects and how side effects impact user trust in the system. The surveys targeted two domains: an autonomous vacuum cleaner and an autonomous vehicle, each with 183 respondents. The results indicate that users are willing to tolerate side effects that are not safety-critical but prefer to minimize them as much as possible. Furthermore, users are willing to assist the system in mitigating negative side effects by providing feedback and reconfiguring the environment. Trust in the system diminishes if it fails to minimize the impacts of negative side effects over time. These results support key fundamental assumptions in existing techniques and facilitate the development of new methods to overcome negative side effects of AI systems.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>
Artificial Intelligence (AI) systems deployed in the open world may produce negative side effects\u2014which are unanticipated, undesirable outcomes that occur in addition to the intended outcomes of the system\u2019s actions. These negative side effects affect users directly or indirectly, by violating their preferences or altering their environment in an undesirable, potentially harmful, manner. While the existing literature has started to explore techniques to overcome the impacts of negative side effects in deployed systems, there has been no prior efforts to determine how users perceive and respond to negative side effects. We surveyed 183 participants to develop an understanding of user attitudes towards side effects and how side effects impact user trust in the system. The surveys targeted two domains: an autonomous vacuum cleaner and an autonomous vehicle, each with 183 respondents. The results indicate that users are willing to tolerate side effects that are not safety-critical but prefer to minimize them as much as possible. Furthermore, users are willing to assist the system in mitigating negative side effects by providing feedback and reconfiguring the environment. Trust in the system diminishes if it fails to minimize the impacts of negative side effects over time. These results support key fundamental assumptions in existing techniques and facilitate the development of new methods to overcome negative side effects of AI systems.<\/div>
Close<\/a><\/p><\/div>

Woolf, Beverly; Ghosh, Aritra; Lan, Andrew; Zilberstein, Shlomo; Juravich, Tom; Cohen, Andrew; Geho, Olivia<\/p>

AI-Enabled Training in Manufacturing Workforce Development Conference<\/span> <\/p>

AAAI Spring Symposium on Artificial Intelligence in Manufacturing, <\/span>2020<\/span>.<\/p>

Abstract<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@conference{SZ:WGLZJCGspring20,
\r\ntitle = {AI-Enabled Training in Manufacturing Workforce Development},
\r\nauthor = {Beverly Woolf and Aritra Ghosh and Andrew Lan and Shlomo Zilberstein and Tom Juravich and Andrew Cohen and Olivia Geho},
\r\nyear  = {2020},
\r\ndate = {2020-01-01},
\r\nbooktitle = {AAAI Spring Symposium on Artificial Intelligence in Manufacturing},
\r\nabstract = {A highly productive workforce can evolve with the integration of digital devices, such as computer interfaces to operating machines, interconnected smart devices, and robots, in the workplace. However, this potential cannot be realized with the current state-of-the-art systems used to train workers. This problem is acute in manufacturing, where huge skills gaps are evident; most workers lack the necessary skills to operate or collaborate with autonomous systems. We propose to address this problem by using intelligent tutoring systems and worker data analysis. The worker data includes: i) fine-grained on-job performance data, ii) career path data containing the entire career paths of workers, and iii) job posting data over a long period of time indicating the required skills for each job. We will collect and analyze worker data and use it to drive new methods for training and reskilling workers. We detail ideas and tools to be developed by research in intelligent tutoring systems, data science, manufacturing, sociology, labor analysis, education, psychology, and economics. We also describe a convergent approach to developing effective, fair, and scalable software solutions and dynamic intelligent training. address = Stanford, California},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {conference}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>

Renski, Henry; Smith-Doerr, Laurel; Wilkerson, Tiamba; Roberts, Shannon C; Zilberstein, Shlomo; Branch, Enobong H<\/p>

Racial Equity and the Future of Work<\/a> Journal Article<\/span> <\/p>

In: <\/span>Technology| Architecture+ Design, <\/span>vol. 4, <\/span>no. 1, <\/span>pp. 17\u201322, <\/span>2020<\/span>.<\/p>

Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@article{SZ:RSWRZBtad20,
\r\ntitle = {Racial Equity and the Future of Work},
\r\nauthor = {Henry Renski and Laurel Smith-Doerr and Tiamba Wilkerson and Shannon C Roberts and Shlomo Zilberstein and Enobong H Branch},
\r\ndoi = {10.1080\/24751448.2020.1705711},
\r\nyear  = {2020},
\r\ndate = {2020-01-01},
\r\njournal = {Technology| Architecture+ Design},
\r\nvolume = {4},
\r\nnumber = {1},
\r\npages = {17--22},
\r\npublisher = {Taylor & Francis doi = 10.1080\/24751448.2020.1705711},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {article}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>