Publications

The Mixture-of-Experts (MoE) architecture has become increasingly popular as a method to scale up large language models (LLMs). To save costs, heterogeneity-aware training solutions have been proposed to utilize GPU clusters made up of both newer and older-generation GPUs. However, existing solutions are agnostic to the performance characteristics of different MoE model components (i.e., attention and expert) and do not fully utilize each GPU's compute capability. In this paper, we introduce HeterMoE, a system to efficiently train MoE models on heterogeneous GPUs. Our key insight is that newer GPUs significantly outperform older generations on attention due to architectural advancements, while older GPUs are still relatively efficient for experts. HeterMoE disaggregates attention and expert computation, where older GPUs are only assigned with expert modules. Through the proposed zebra parallelism, HeterMoE overlaps the computation on different GPUs, in addition to employing an asymmetric expert assignment strategy for fine-grained load balancing to minimize GPU idle time. Our evaluation shows that HeterMoE achieves up to 2.3x speed-up compared to existing MoE training systems, and 1.4x compared to an optimally balanced heterogeneity-aware solution. HeterMoE efficiently utilizes older GPUs by maintaining 95% training throughput on average, even with half of the GPUs in a homogeneous A40 cluster replaced with V100.

Policy Points Accountable care organizations (ACOs) have incentives to promote the adoption of patient engagement strategies such as shared decision making and self‐management support programs to improve patient outcomes and contain health care costs. High adoption of patient engagement strategies among ACO‐affiliated practices did not improve patient‐reported outcomes (PROs) of physical, emotional, and social function among adult patients with diabetes and/or cardiovascular disease over a one‐year time frame, likely because implementing these strategies requires extensive clinician and staff training, workflow redesign, and patient participation over time. A dominant focus on improving clinical measures to meet external requirements may crowd out time needed for care team members to address other outcomes that matter to patients, including PROs. Payers and policy‐makers should explicitly incentivize the collection and use of PROs when contracting with ACOs. Context Adult primary care practices of accountable care organizations (ACOs) are adopting a range of patient engagement strategies, but little is known about how these strategies are related to patient‐reported outcomes (PROs) and how relational coordination among team members aids implementation. Methods We used a mixed‐methods cohort study design integrating administrative and clinical data with two data collection waves (2014‐2015 and 2016‐2017) of clinician and staff surveys ( n = 764), surveys of adult patients with diabetes and/or cardiovascular disease (CVD) ( n = 1,276), and key informant interviews of clinicians, staff, and administrators ( n = 103). Multivariable linear regression estimated the relationship of practice adoption of patient engagement strategies, relational coordination, and PROs of physical, social, and emotional function. The mediating role of patient activation was examined using cross‐lagged panel models. Key informant interviews assessed how relational coordination influences the implementation of patient engagement strategies. Findings There were no differential improvements in PROs among patients of practices with high vs. low adoption of patient engagement strategies or among patients of practices with high vs. low relational coordination. The Patient Activation Measure (PAM) is strongly related to better physical, emotional, and social PROs over time. Relational coordination facilitated the implementation of patient engagement strategies, but key informants indicated that resources and systems to systematically track treatment preferences and goals beyond clinical indicators were needed to support effective implementation. Conclusions Adult patients with diabetes and/or CVD of ACO‐affiliated practices with high adoption of patient engagement strategies do not have improved PROs of physical, emotional, and social function over a one‐year time frame. Implementing patient engagement strategies increases task interdependence among primary care team members, which needs to be carefully managed. ACOs may need to make greater investment in collecting, monitoring, and analyzing PRO data to ensure that practice adoption and implementation of patient engagement strategies leads to improved physical, emotional, and social function among patients.

Human movement is goal-directed and influenced by the spatial layout of the objects in the scene. To plan future human motion, it is crucial to perceive the environment -- imagine how hard it is to navigate a new room with lights off. Existing works on predicting human motion do not pay attention to the scene context and thus struggle in long-term prediction. In this work, we propose a novel three-stage framework that exploits scene context to tackle this task. Given a single scene image and 2D pose histories, our method first samples multiple human motion goals, then plans 3D human paths towards each goal, and finally predicts 3D human pose sequences following each path. For stable training and rigorous evaluation, we contribute a diverse synthetic dataset with clean annotations. In both synthetic and real datasets, our method shows consistent quantitative and qualitative improvements over existing methods.

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures.

Overfishing and environmental change have triggered many severe and unexpected consequences. As existing communities have collapsed, new ones have become established, fundamentally transforming ecosystems to those that are often less productive for fisheries, more prone to cycles of booms and busts, and thus less manageable. We contend that the failure of fisheries science and management to anticipate these transformations results from a lack of appreciation for the nature, strength, complexity, and outcome of species interactions. Ecologists have come to understand that networks of interacting species exhibit nonlinear dynamics and feedback loops that can produce sudden and unexpected shifts. We argue that fisheries science and management must follow this lead by developing a sharper focus on species interactions and how disrupting these interactions can push ecosystems in which fisheries are embedded past their tipping points.

Block copolymer patterned holey silicon (HS) was successfully integrated into a microdevice for simultaneous measurements of Seebeck coefficient, electrical conductivity, and thermal conductivity of the same HS microribbon. These fully integrated HS microdevices provided excellent platforms for the systematic investigation of thermoelectric transport properties tailored by the dimensions of the periodic hole array, that is, neck and pitch size, and the doping concentrations. Specifically, thermoelectric transport properties of HS with a neck size in the range of 16-34 nm and a fixed pitch size of 60 nm were characterized, and a clear neck size dependency was shown in the doping range of 3.1 × 10(18) to 6.5 × 10(19) cm(-3). At 300 K, thermal conductivity as low as 1.8 ± 0.2 W/mK was found in HS with a neck size of 16 nm, while optimized zT values were shown in HS with a neck size of 24 nm. The controllable effects of holey array dimensions and doping concentrations on HS thermoelectric performance could aid in improving the understanding of the phonon scattering process in a holey structure and also in facilitating the development of silicon-based thermoelectric devices.