Artificial Intelligence (AI) is a rapidly advancing field. Novel machine learning methods combined with reasoning and search techniques have led us to reach new milestones with increasing frequency, from self-driving cars to computer vision, machine translation, computer Go trained on human play, to Go and Chess world-champion level play using pure self-training strategies. These ever expanding AI capabilities open up exciting new avenues for automating scientific discovery. I will discuss our work on using AI for accelerating and automating materials discovery. In particular, we have focused on high-throughput structure determination for combinatorial materials discovery and on solving the phase map diagram problem for composition libraries. While standard statistical and machine learning methods are important to address this challenge, they fail to incorporate relationships arising from the physics of the underlying materials. I will introduce an effective approach based on a tight integration of machine learning methods, to deal with noise and uncertainty in the measurement data, with optimization and inference techniques, to incorporate the rich set of constraints arising from the underlying physics. Finally, I will describe our vision for a Scientific Autonomous Reasoning Agent (SARA), a multi-Agent system to accelerate materials discovery integrating in a synergistic and complementary way, first principles quantum physics, experimental materials synthesis, processing, and characterization, and AI based algorithms for reasoning and scientific discovery, including the representation, planning, optimization, and learning of materials knowledge.