Publications

State-of-the-art sparse attention methods for reducing decoding latency fall into two main categories: approximate top-$k$ (and its extension, top-$p$) and recently introduced sampling-based estimation. However, these approaches are fundamentally limited in their ability to approximate full attention: they fail to provide consistent approximations across heads and query vectors and, most critically, lack guarantees on approximation quality, limiting their practical deployment. We observe that top-$k$ and random sampling are complementary: top-$k$ performs well when attention scores are dominated by a few tokens, whereas random sampling provides better estimates when attention scores are relatively uniform. Building on this insight and leveraging the statistical guarantees of sampling, we introduce vAttention, the first practical sparse attention mechanism with user-specified $(ε, δ)$ guarantees on approximation accuracy (thus, verified). These guarantees make vAttention a compelling step toward practical, reliable deployment of sparse attention at scale. By unifying top-k and sampling, vAttention outperforms both individually, delivering a superior quality-efficiency trade-off. Our experiments show that vAttention significantly improves the quality of sparse attention (e.g., $\sim$4.5 percentage points for Llama-3.1-8B-Inst and Deepseek-R1-Distill-Llama-8B on RULER-HARD), and effectively bridges the gap between full and sparse attention (e.g., across datasets, it matches full model quality with upto 20x sparsity). We also demonstrate that it can be deployed in reasoning scenarios to achieve fast decoding without compromising model quality (e.g., vAttention achieves full model quality on AIME2024 at 10x sparsity with up to 32K token generations). Code is open-sourced at https://github.com/xAlg-ai/sparse-attention-hub.

The CO 2 reduction reaction (CO2RR) is a complex process with many possible products, and maintaining high selectivity for multi-carbon products remains a challenge. Currently, Cu-based catalysts are the only transition metal catalysts that can produce C2 products, and one of the popular Cu-based catalysts is CuO nanowires (NWs). We studied a series of CuO nanowires ranging from 50 to 200 nm for CO2RR using multiple characterization techniques including transmission electron microscopy(TEM), small angle X-ray scattering (SAXS) and in situ X-ray adsorption spectroscopy. We found that there could be an activation process required for such CuO NWs to serve as efficient CO2RR catalysts. The CuO can go through multiple steps resulting in the severe morphology changes during the activation process, confirmed by TEM and SAXS. In situ XAS results showed that the CuO NWs will be reduced from Cu(II) to Cu(0) during the reaction, consistent with our previous work(1). H-cell testing results also indicated the C2 production can be boosted after specific activation pathway, which highlights the importance of catalyst pre-treatment. These findings not only deepen our understanding of CO2RR mechanism but also provide fundamental insights for CO2RR catalysts synthesis. References (1) Soo Hong Lee, Ian Sullivan, David M. Larson, Guiji Liu, Francesca M. Toma, Chengxiang Xiang, and Walter S. Drisdell. ACS Catalysis, 2020, 10 (14), 8000-8011 Acknowledgement The authors acknowledge the funding support from the Laboratory Directed Research and Development Program of the Lawrence Berkeley National Laboratory under the U.S. Department of Energy Contract (No. DE-AC02-05CH11231).

We report the detection of a faint optical flash by the 2-m Faulkes Telescope North simultaneously with the second of two prompt γ-ray pulses in INTEGRAL gamma-ray burst (GRB) 080603A, beginning at trest= 37 s after the onset of the GRB. This optical flash appears to be distinct from the subsequent emerging afterglow emission, for which we present comprehensive broad-band radio to X-ray light curves to 13 d post-burst and rigorously test the standard fireball model. The intrinsic extinction towards GRB 080603A is high (AV, z= 0.8 mag), and the well-sampled X-ray-to-near-infrared spectral energy distribution is interesting in requiring an LMC2 extinction profile, in contrast to the majority of GRBs. Comparison of the γ-ray and extinction-corrected optical flux densities of the flash rules out an inverse-Compton origin for the prompt γ-rays; instead, we suggest that the optical flash could originate from the inhomogeneity of the relativistic flow. In this scenario, a large velocity irregularity in the flow produces the prompt γ-rays, followed by a milder internal shock at a larger radius that would cause the optical flash. Flat γ-ray spectra, roughly F∝ν−0.1, are observed in many GRBs. If the flat spectrum extends down to the optical band in GRB 080603A, the optical flare could be explained as the low-energy tail of the γ-ray emission. If this is indeed the case, it provides an important clue to understanding the nature of the emission process in the prompt phase of GRBs and highlights the importance of deep (R > 20 mag), rapid follow-up observations capable of detecting faint, prompt optical emission.

Abstract The development of a gold(I)‐catalyzed sulfination of aryl boronic acids is described. This transformation proceeds through an unprecedented mechanism which exploits the reactivity of gold(I)–heteroatom bonds to form sulfinate anions. Further in situ elaboration of the sulfinate intermediates leads to the corresponding sulfones and sulfonamides, two pharmacophores routinely encountered in drug discovery.

An entry from the Cambridge Structural Database, the world’s repository for small molecule crystal structures. The entry contains experimental data from a crystal diffraction study. The deposited dataset for this entry is freely available from the CCDC and typically includes 3D coordinates, cell parameters, space group, experimental conditions and quality measures.