Publications

Evaluating the reasoning abilities of large language models (LLMs) is challenging. Existing benchmarks often depend on static datasets, which are vulnerable to data contamination and may get saturated over time, or on binary live human feedback that conflates reasoning with other abilities. As the most prominent dynamic benchmark, Chatbot Arena evaluates open-ended questions in real-world settings, but lacks the granularity in assessing specific reasoning capabilities. We introduce GameArena, a dynamic benchmark designed to evaluate LLM reasoning capabilities through interactive gameplay with humans. GameArena consists of three games designed to test specific reasoning capabilities (e.g., deductive and inductive reasoning), while keeping participants entertained and engaged. We analyze the gaming data retrospectively to uncover the underlying reasoning processes of LLMs and measure their fine-grained reasoning capabilities. We collect over 2000 game sessions and provide detailed assessments of various reasoning capabilities for five state-of-the-art LLMs. Our user study with 100 participants suggests that GameArena improves user engagement compared to Chatbot Arena. For the first time, GameArena enables the collection of step-by-step LLM reasoning data in the wild.

Using electron-beam-induced deposition and focused-ion-beam milling, we have fabricated and demonstrated a nanofocusing optical transformer with a 3-dimensionally tapered tip. At the tip, the light is confined to 13-by-80-nm area with intensity enhancement exceeding 1500.

Comparing photosynthetic and photovoltaic efficiencies is not a simple issue. Although both processes harvest the energy in sunlight, they operate in distinctly different ways and produce different types of products: biomass or chemical fuels in the case of natural photosynthesis and nonstored electrical current in the case of photovoltaics. In order to find common ground for evaluating energy-conversion efficiency, we compare natural photosynthesis with present technologies for photovoltaic-driven electrolysis of water to produce hydrogen. Photovoltaic-driven electrolysis is the more efficient process when measured on an annual basis, yet short-term yields for photosynthetic conversion under optimal conditions come within a factor of 2 or 3 of the photovoltaic benchmark. We consider opportunities in which the frontiers of synthetic biology might be used to enhance natural photosynthesis for improved solar energy conversion efficiency.

To analyze the effect of lithium and microstructure on the pitting corrosion behavior of aluminum alloys, three types of aluminum alloys were studied via scanning electron microscopy, transmission electron microscopy, electrochemical polarization, and by immersion tests coupled with in-situ observation of pitting and statistical analysis of pit depths measured by surface profilometry. It was found that, with increasing lithium content, the resistance to pitting corrosion was enhanced and the passive range was enlarged. In-situ observation revealed that the development of pitting corrosion exhibited three stages, including an initial slow nucleation stage (Stage I), a fast development stage (Stage II), and a stabilized growth stage (Stage III). Higher lithium content contributed to shorter time periods of Stages I and II, resulting in faster pitting evolution and a higher number of pits. However, the pits were generally shallower for the specimen with the highest lithium content, which is in agreement with the results of the electrochemical analysis.

Nanoscale patterning and electronics fabrication on flexible polymer substrates via direct nanoirnprinting of metal nanoparticles was demonstrated. The metal nanoparticle solution enables a simple one-step metal nanoirnprinting with very low pressure and temperature. Electrical and structural characterizations on flexible substrate during cyclic bending deformations verify the robustness of the nanoimprinted structures to mechanical deformation conditions.

The emergence of a quantum spin liquid (QSL), a state of matter that can result when electron spins are highly correlated but do not become ordered, has been the subject of a considerable body of research in condensed matter physics. Spin liquid states have been proposed as hosts for high-temperature superconductivity and can host topological properties with potential applications in quantum information science. The excitations of most quantum spin liquids are not conventional spin waves but rather quasiparticles known as spinons, whose existence is well established experimentally only in one-dimensional systems; the unambiguous experimental realization of QSL behavior in higher dimensions remains challenging. Here, we investigate the novel compound YbZn2GaO5, which hosts an ideal triangular lattice of effective spin-1/2 moments with no detectable inherent chemical disorder. Thermodynamic and inelastic neutron scattering measurements performed on high-quality single crystal samples of YbZn2GaO5 exclude the possibility of long-range magnetic ordering down to 0.06 K, demonstrate a quadratic power law for the specific heat and reveal a continuum of magnetic excitations in parts of the Brillouin zone. Both low-temperature thermodynamics and inelastic neutron scattering spectra suggest that YbZn2GaO5 is a U(1) Dirac QSL with spinon excitations concentrated at certain points in the Brillouin zone. We advanced these results by performing additional specific heat measurements under finite fields, further confirming the theoretical expectations for a Dirac QSL on the triangular lattice of YbZn2GaO5.

Finding new compounds and their crystal structures is an essential step to new materials discoveries. We demonstrate how this search can be accelerated using a combination of machine learning techniques and high-throughput ab initio computations. Using a probabilistic model built on an experimental crystal structure database, novel compositions that are most likely to form a compound, and their most-probable crystal structures, are identified and tested for stability by ab initio computations. We performed such a large-scale search for new ternary oxides, discovering 209 new compounds with a limited computational budget. A list of these predicted compounds is provided, and we discuss the chemistries in which high discovery rates can be expected.