Publications

In Large Language Model (LLM) inference, the output length of an LLM request is typically regarded as not known a priori. Consequently, most LLM serving systems employ a simple First-come-first-serve (FCFS) scheduling strategy, leading to Head-Of-Line (HOL) blocking and reduced throughput and service quality. In this paper, we reexamine this assumption -- we show that, although predicting the exact generation length of each request is infeasible, it is possible to predict the relative ranks of output lengths in a batch of requests, using learning to rank. The ranking information offers valuable guidance for scheduling requests. Building on this insight, we develop a novel scheduler for LLM inference and serving that can approximate the shortest-job-first (SJF) schedule better than existing approaches. We integrate this scheduler with the state-of-the-art LLM serving system and show significant performance improvement in several important applications: 2.8x lower latency in chatbot serving and 6.5x higher throughput in synthetic data generation. Our code is available at https://github.com/hao-ai-lab/vllm-ltr.git

The scales of the arapaima (Arapaima gigas), one of the largest freshwater fish in the world, can serve as inspiration for the design of flexible dermal armor. Each scale is composed of two layers: a laminate composite of parallel collagen fibrils and a hard, highly mineralized surface layer. We review the structure of the arapaima scales and examine the functions of the different layers, focusing on the mechanical behavior, including tension and penetration of the scales, with and without the highly mineralized outer layer. We show that the fracture of the mineral and the stretching, rotation and delamination of collagen fibrils dissipate a significant amount of energy prior to catastrophic failure, providing high toughness and resistance to penetration by predator teeth. We show that the arapaima’s scale has evolved to minimize damage from penetration by predator teeth through a Bouligand-like arrangement of successive layers, each consisting of parallel collagen fibrils with different orientations. This inhibits crack propagation and restricts damage to an area adjoining the penetration. The flexibility of the lamellae is instrumental to the redistribution of the compressive stresses in the underlying tissue, decreasing the severity of the concentrated load produced by the action of a tooth. The experimental results, combined with small-angle X-ray scattering characterization and molecular dynamics simulations, provide a complete picture of the mechanisms of deformation, delamination and rotation of the lamellae during tensile extension of the scale.

We report the discovery of a transient seen in a strongly lensed arc at redshift $z_{\rm s}=1.2567$ in \emph{Hubble Space Telescope} imaging of the Abell 370 galaxy cluster. The transient is detected at $29.51\pm0.14$ AB mag in a WFC3/UVIS F200LP difference image made using observations from two different epochs, obtained in the framework of the \emph{Flashlights} program, and is also visible in the F350LP band ($m_{\rm F350LP} \approx 30.53\pm0.76$ AB mag). The transient is observed on the negative-parity side of the critical curve at a distance of $\sim 0.6"$ from it, greater than previous examples of lensed stars. The large distance from the critical curve yields a significantly smaller macromagnification, but our simulations show that bright, O/B-type supergiants can reach sufficiently high magnifications to be seen at the observed position and magnitude. In addition, the observed transient image is a trailing image with an observer-frame time delay of $\sim+0.8$ days from its expected counterpart, so that any transient lasting for longer than that should have also been seen on the minima side and is thus excluded. This, together with the blue colour we measure for the transient ($m_{\rm F200LP} - m_{\rm F350LP} \approx [-0.3,-1.6]$ AB), rules out most other transient candidates such as (kilo)novae, for example, and makes a lensed star the prime candidate. Assuming the transient is indeed a lensed star as suggested, many more such events should be detected in the near future in cluster surveys with the \emph{Hubble Space Telescope} and \emph{James Webb Space Telescope}.

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures.

Recent progress in colloidal synthesis of nanoparticles with well-controlled size, shape, and composition, together with development of in situ surface science characterization tools, such as ambient pressure X-ray photoelectron spectroscopy (APXPS), has generated new opportunities to unravel the surface structure of working catalysts. We report an APXPS study of Ru nanoparticles to investigate catalytically active species on Ru nanoparticles under oxidizing, reducing, and CO oxidation reaction conditions. The 2.8 and 6 nm Ru nanoparticle model catalysts were synthesized in the presence of poly(vinyl pyrrolidone) polymer capping agent and deposited onto a flat Si support as two-dimensional arrays using the Langmuir-Blodgett deposition technique. Mild oxidative and reductive characteristics indicate the formation of surface oxide on the Ru nanoparticles, the thickness of which is found to be dependent on nanoparticle size. The larger 6 nm Ru nanoparticles were oxidized to a smaller extent than the smaller Ru 2.8 nm nanoparticles within the temperature range of 50-200 °C under reaction conditions, which appears to be correlated with the higher catalytic activity of the bigger nanoparticles. We found that the smaller Ru nanoparticles form bulk RuO(2) on their surfaces, causing the lower catalytic activity. As the size of the nanoparticle increases, the core-shell type RuO(2) becomes stable. Such in situ observations of Ru nanoparticles are useful in identifying the active state of the catalysts during use and, hence, may allow for rational catalyst designs for practical applications.

<p>H & E staining of 22Rv1-DR xenograft tumor tissues from mice treated with Vehicle control, docetaxel alone, DSF/CuCl2, and docetaxel plus DSF/CuCl2.</p>

Multimetallic nanowires with long-range atomic ordering hold the promise of unique physicochemical properties in many applications. Here we demonstrate the synthesis and study the stability of Cu3Au intermetallic nanowires. The synthesis is achieved by using Cu@Au core-shell nanowires as precursors. With appropriate Cu/Au stoichiometry, the Cu@Au core-shell nanowires are transformed into fully ordered Cu3Au nanowires under thermal annealing. Thermally-driven atom diffusion accounts for this transformation as revealed by X-ray diffraction and electron microscopy studies. The twin boundaries abundant in the Cu@Au core-shell nanowires facilitate the ordering process. The resulting Cu3Au intermetallic nanowires have uniform and accurate atomic positioning in the crystal lattice, which enhances the nobility of Cu. No obvious copper oxides are observed in fully ordered Cu3Au nanowires after annealing in air at 200 °C, a temperature that is much higher than those observed in Cu@Au core-shell and pure Cu nanowires. This work opens up an opportunity for further research into the development and applications of intermetallic nanowires. [Figure not available: see fulltext.].