-
X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Design
Authors:
Eric L. Buehler,
Markus J. Buehler
Abstract:
We report a mixture of expert strategy to create fine-tuned large language models using a deep layer-wise token-level approach based on low-rank adaptation (LoRA). Starting with a set of pre-trained LoRA adapters, our gating strategy uses the hidden states to dynamically mix adapted layers, allowing the resulting X-LoRA model to draw upon different capabilities and create never-before-used deep la…
▽ More
We report a mixture of expert strategy to create fine-tuned large language models using a deep layer-wise token-level approach based on low-rank adaptation (LoRA). Starting with a set of pre-trained LoRA adapters, our gating strategy uses the hidden states to dynamically mix adapted layers, allowing the resulting X-LoRA model to draw upon different capabilities and create never-before-used deep layer-wise combinations to solve tasks. The design is inspired by the biological principles of universality and diversity, where neural network building blocks are reused in different hierarchical manifestations. Hence, the X-LoRA model can be easily implemented for any existing large language model (LLM) without a need for modifications of the underlying structure. We develop a tailored X-LoRA model that offers scientific capabilities including forward/inverse analysis tasks and enhanced reasoning capability, focused on biomaterial analysis, protein mechanics and design. The impact of this work include access to readily expandable and adaptable models with strong domain knowledge and the capability to integrate across areas of knowledge. Featuring experts in biology, mathematics, reasoning, bio-inspired materials, mechanics and materials, chemistry, protein biophysics, mechanics and quantum-mechanics based molecular properties, we conduct a series of physics-focused case studies. We examine knowledge recall, protein mechanics forward/inverse tasks, protein design, adversarial agentic modeling including ontological knowledge graph construction, as well as molecular design. The model is capable not only of making quantitative predictions of nanomechanical properties of proteins or quantum mechanical molecular properties, but also reasons over the results and correctly predicts likely mechanisms that explain distinct molecular behaviors.
△ Less
Submitted 30 March, 2024; v1 submitted 11 February, 2024;
originally announced February 2024.
-
LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexia
Authors:
Steven M. Goodman,
Erin Buehler,
Patrick Clary,
Andy Coenen,
Aaron Donsbach,
Tiffanie N. Horne,
Michal Lahav,
Robert Macdonald,
Rain Breaw Michaels,
Ajit Narayanan,
Mahima Pushkarna,
Joel Riley,
Alex Santana,
Lei Shi,
Rachel Sweeney,
Phil Weaver,
Ann Yuan,
Meredith Ringel Morris
Abstract:
Prior work has explored the writing challenges experienced by people with dyslexia, and the potential for new spelling, grammar, and word retrieval technologies to address these challenges. However, the capabilities for natural language generation demonstrated by the latest class of large language models (LLMs) highlight an opportunity to explore new forms of human-AI writing support tools. In thi…
▽ More
Prior work has explored the writing challenges experienced by people with dyslexia, and the potential for new spelling, grammar, and word retrieval technologies to address these challenges. However, the capabilities for natural language generation demonstrated by the latest class of large language models (LLMs) highlight an opportunity to explore new forms of human-AI writing support tools. In this paper, we introduce LaMPost, a prototype email-writing interface that explores the potential for LLMs to power writing support tools that address the varied needs of people with dyslexia. LaMPost draws from our understanding of these needs and introduces novel AI-powered features for email-writing, including: outlining main ideas, generating a subject line, suggesting changes, rewriting a selection. We evaluated LaMPost with 19 adults with dyslexia, identifying many promising routes for further exploration (including the popularity of the "rewrite" and "subject line" features), but also finding that the current generation of LLMs may not surpass the accuracy and quality thresholds required to meet the needs of writers with dyslexia. Surprisingly, we found that participants' awareness of the AI had no effect on their perception of the system, nor on their feelings of autonomy, expression, and self-efficacy when writing emails. Our findings yield further insight into the benefits and drawbacks of using LLMs as writing support for adults with dyslexia and provide a foundation to build upon in future research.
△ Less
Submitted 5 July, 2022;
originally announced July 2022.
-
Spared cognitive and behavioral functions prior to epilepsy onset in a rat model of 2 subcortical band heteropia
Authors:
Fanny Sandrine Martineau,
Lauriane Fournier,
Emmanuelle Buhler,
Françoise Watrin,
Francesca Sargolini,
Jean-Bernard Manent,
Bruno Poucet,
Alfonso Represa
Abstract:
13 Subcortical band heterotopia (SBH), also known as doublecortex syndrome, is a 14 malformation of cortical development resulting from mutations in the doublecortin gene 15 (DCX). It is characterized by a lack of migration of cortical neurons that accumulate in the 16 white matter forming a heterotopic band. Patients with SBH may present mild to moderate 17 intellectual disability as well as epil…
▽ More
13 Subcortical band heterotopia (SBH), also known as doublecortex syndrome, is a 14 malformation of cortical development resulting from mutations in the doublecortin gene 15 (DCX). It is characterized by a lack of migration of cortical neurons that accumulate in the 16 white matter forming a heterotopic band. Patients with SBH may present mild to moderate 17 intellectual disability as well as epilepsy. The SBH condition can be modeled in rats by in 18 utero knockdown (KD) of Dcx. The affected cells form an SBH reminiscent of that observed in 19 human patients and the animals develop a chronic epileptic condition in adulthood. Here, 20 we investigated if the presence of an SBH is sufficient to induce cognitive impairment in 21
△ Less
Submitted 1 March, 2019;
originally announced March 2019.
-
Role of the ratio of biopolyelectrolyte persistence length to nanoparticle size in the structural tuning of electrostatic complexes
Authors:
Li Shi,
Florent Carn,
François Boué,
Eric Buhler
Abstract:
Aggregation of nanoparticles of given size $R$ induced by addition of a polymer strongly depends on its degree of rigidity. This is shown here on a large variety of silica nanoparticle self-assemblies obtained by electrostatic complexation with carefully selected oppositely charged bio-polyelectrolytes of different rigidity. The effective rigidity is quantified by the total persistence length…
▽ More
Aggregation of nanoparticles of given size $R$ induced by addition of a polymer strongly depends on its degree of rigidity. This is shown here on a large variety of silica nanoparticle self-assemblies obtained by electrostatic complexation with carefully selected oppositely charged bio-polyelectrolytes of different rigidity. The effective rigidity is quantified by the total persistence length $L_T$ representing the sum of the intrinsic ($L_p$) and electrostatic ($L_e$) polyelectrolyte persistence length, which depends on the screening, i.e., on ionic strength due to counter-ions and external salt concentrations. We experimentally show for the first time that the ratio L T /R is the main tuning parameter that controls the fractal dimension D f of the nanoparticles self-assemblies, which is determined using small-angle neutron scattering: (i) For $L_T /R<0.3$ (obtained with flexible poly-L-lysine in the presence of an excess of salt), chain flexibility promotes easy wrapping around nanoparticles in excess hence ramified structures with $D_f \sim 2$. (ii) For $0.3<L_T /R\le1$ (semiflexible chitosan or hyaluronan complexes), chain stiffness promotes the formation of one-dimensional nanorods (in excess of nanoparticles), in good agreement with computer simulations. (iii) For $L_T /R>1$, $L_e$ is strongly increased due to the absence of salt and repulsions between nanoparticles cannot be compensated by the polyelectrolyte wrapping, which allow a spacing between nanoparticles and the formation of one dimensional pearl necklace complexes. (iv) Finally, electrostatic 2 screening, i.e. ionic strength, turned out to be a reliable way of controlling $D_f$ and the phase diagram behavior. It finely tunes the short-range interparticle potential, resulting in larger fractal dimensions at higher ionic strength.
△ Less
Submitted 7 November, 2016;
originally announced November 2016.
-
New Insight Into the Size Tuning of Monodispersed Colloidal Gold Obtained by Citrate Method
Authors:
Li Shi,
Eric Buhler,
François Boué,
Florent Carn
Abstract:
We study the effect of citrate to gold molar ratio (X) on the size of citrated gold nanoparticles (AuNPs). This dependence is still a matter of debate for X $\ge$ 3 where the polydispersity is yet minimized. Indeed, there is no consensus between experiments proposed so far for comparable experimental conditions. Nonetheless, the sole available theoretical prediction has never been validated experi…
▽ More
We study the effect of citrate to gold molar ratio (X) on the size of citrated gold nanoparticles (AuNPs). This dependence is still a matter of debate for X $\ge$ 3 where the polydispersity is yet minimized. Indeed, there is no consensus between experiments proposed so far for comparable experimental conditions. Nonetheless, the sole available theoretical prediction has never been validated experimentally in this range of X. We show unambiguously using 3 techniques (UV-Vis spectroscopy, dynamic light scattering and transmission electronic microscopy), 2 different synthetic approaches (Direct, Inverse) and 10 X values for each approach that AuNPs' size decay as a monoexponential with X. This result is, for the first time, in agreement with the sole available theoretical prediction by Kumar et al. on the whole studied range of X.
△ Less
Submitted 3 November, 2016;
originally announced November 2016.
-
Receding-horizon Stochastic Model Predictive Control with Hard Input Constraints and Joint State Chance Constraints
Authors:
Joel A. Paulson,
Edward A. Buehler,
Richard D. Braatz,
Ali Mesbah
Abstract:
This article considers the stochastic optimal control of discrete-time linear systems subject to (possibly) unbounded stochastic disturbances, hard constraints on the manipulated variables, and joint chance constraints on the states. A tractable convex second-order cone program (SOCP) is derived for calculating the receding-horizon control law at each time step. Feedback is incorporated during pre…
▽ More
This article considers the stochastic optimal control of discrete-time linear systems subject to (possibly) unbounded stochastic disturbances, hard constraints on the manipulated variables, and joint chance constraints on the states. A tractable convex second-order cone program (SOCP) is derived for calculating the receding-horizon control law at each time step. Feedback is incorporated during prediction by parametrizing the control law as an affine function of the disturbances. Hard input constraints are guaranteed by saturating the disturbances that appear in the control law parametrization. The joint state chance constraints are conservatively approximated as a collection of individual chance constraints that are subsequently relaxed via the Cantelli-Chebyshev inequality. Feasibility of the SOCP is guaranteed by softening the approximated chance constraints using the exact penalty function method. Closed-loop stability in a stochastic sense is established by establishing that the states satisfy a geometric drift condition outside of a compact set such that their variance is bounded at all times. The SMPC approach is demonstrated using a continuous acetone-butanol-ethanol fermentation process, which is used for production of high-value-added drop-in biofuels.
△ Less
Submitted 28 June, 2015;
originally announced June 2015.
-
Lyapunov-based Stochastic Nonlinear Model Predictive Control: Shaping the State Probability Density Functions
Authors:
Edward A. Buehler,
Joel A. Paulson,
Ali Akhavan,
Ali Mesbah
Abstract:
Stochastic uncertainties in complex dynamical systems lead to variability of system states, which can in turn degrade the closed-loop performance. This paper presents a stochastic model predictive control approach for a class of nonlinear systems with unbounded stochastic uncertainties. The control approach aims to shape probability density function of the stochastic states, while satisfying input…
▽ More
Stochastic uncertainties in complex dynamical systems lead to variability of system states, which can in turn degrade the closed-loop performance. This paper presents a stochastic model predictive control approach for a class of nonlinear systems with unbounded stochastic uncertainties. The control approach aims to shape probability density function of the stochastic states, while satisfying input and joint state chance constraints. Closed-loop stability is ensured by designing a stability constraint in terms of a stochastic control Lyapunov function, which explicitly characterizes stability in a probabilistic sense. The Fokker-Planck equation is used for describing the dynamic evolution of the states' probability density functions. Complete characterization of probability density functions using the Fokker-Planck equation allows for shaping the states' density functions as well as direct computation of joint state chance constraints. The closed-loop performance of the stochastic control approach is demonstrated using a continuous stirred-tank reactor.
△ Less
Submitted 12 May, 2015;
originally announced May 2015.
-
Nanorods of Well-Defined Length and Monodisperse Cross-Section Obtained from Electrostatic Complexation of Nanoparticles with a Semiflexible Biopolymer
Authors:
Li Shi,
Florent Carn,
François Boué,
Gervaise Mosser,
Eric Buhler
Abstract:
We show by combining small-angle X-ray scattering (SAXS) and cryo-transmission electron microscopy (cryo-TEM) that anionic silica nanoparticles (SiNPs) assemble into well-defined 1D cluster when mixed with a dilute solution of semiflexible chitosan polycation. The nanorods are stable in excess of SiNPs and composed of 10 SiNPs well-ordered into straight single strands with length Lrod \approx 184.…
▽ More
We show by combining small-angle X-ray scattering (SAXS) and cryo-transmission electron microscopy (cryo-TEM) that anionic silica nanoparticles (SiNPs) assemble into well-defined 1D cluster when mixed with a dilute solution of semiflexible chitosan polycation. The nanorods are stable in excess of SiNPs and composed of 10 SiNPs well-ordered into straight single strands with length Lrod \approx 184.0 nm and radius Rrod = 9.2 nm = RSiNPs. We point out that the ratio between the chitosan persistence length and the SiNP radius, which is here equal to 1, can be the determining condition to obtain such original objects.
△ Less
Submitted 21 October, 2012;
originally announced October 2012.
-
Rodlike Complexes of a Polyelectrolyte (Hyaluronan) and a Protein (Lysozyme) observed by SANS
Authors:
François Boué,
Eric Buhler,
Fabrice Cousin,
Isabelle Grillo,
Isabelle Morfin
Abstract:
We study by Small Angle Neutron Scattering (SANS) the structure of Hyaluronan -Lysozyme complexes. Hyaluronan (HA) is a polysaccharide of 9 nm intrinsic persistence length that bears one negative charge per disaccharide monomer (Mmol = 401.3 g/mol); two molecular weights, Mw = 6000 and 500 000 Da were used. The pH was adjusted at 4.7 and 7.4 so that lysozyme has a global charge of +10 and + 8 resp…
▽ More
We study by Small Angle Neutron Scattering (SANS) the structure of Hyaluronan -Lysozyme complexes. Hyaluronan (HA) is a polysaccharide of 9 nm intrinsic persistence length that bears one negative charge per disaccharide monomer (Mmol = 401.3 g/mol); two molecular weights, Mw = 6000 and 500 000 Da were used. The pH was adjusted at 4.7 and 7.4 so that lysozyme has a global charge of +10 and + 8 respectively. The lysozyme concentration was varied from 3 to 40 g/L, at constant HA concentration (10 g/L). At low protein concentration, samples are monophasic and SANS experiments reveal only fluctuations of concentration although, at high protein concentration, clusters are observed by SANS in the dense phase of the diphasic samples. In between, close to the onset of the phase separation, a distinct original scattering is observed. It is characteristic of a rod-like shape, which could characterize "single" complexes involving one or a few polymer chains. For the large molecular weight (500 000) the rodlike rigid domains extend to much larger length scale than the persistence length of the HA chain alone in solution and the range of the SANS investigation. They can be described as a necklace of proteins attached along a backbone of diameter one or a few HA chains. For the short chains (Mw ~ 6000), the rod length of the complexes is close to the chain contour length (~ 15 nm).
△ Less
Submitted 3 October, 2012;
originally announced October 2012.
-
Suppression of Aggregation in Natural-Semiflexible/Flexible Polyanion Mixtures, and Direct Check of the OSF Model using SANS
Authors:
Fabien Bonnet,
Ralph Schweins,
François Boué,
Eric Buhler
Abstract:
Aggregation and other interactions are suppressed for a biological semiflexible polyelectrolyte, hyaluronan (HA), when it is embedded in a mixture with another negatively charged and flexible polyelectrolyte chain, sodium polystyrene sulfonate. We see directly HA only in the mixture using Small-Angle Neutron Scattering, isotopic labelling and contrast matching. At low ionic strength, for which a…
▽ More
Aggregation and other interactions are suppressed for a biological semiflexible polyelectrolyte, hyaluronan (HA), when it is embedded in a mixture with another negatively charged and flexible polyelectrolyte chain, sodium polystyrene sulfonate. We see directly HA only in the mixture using Small-Angle Neutron Scattering, isotopic labelling and contrast matching. At low ionic strength, for which aggregation is usually seen for pure HA solutions, an unambiguous set of experimental results shows that we neither observe HA aggregation nor a polyelectrolyte peak (observed for solutions of single species); instead we observe a wormlike chain behaviour characteristic of single chain with a variation of the persistence length with the square of the Debye screening length, Le~κ^-2, as formerly predicted by Odijk and not yet observed on a polymer chain.
△ Less
Submitted 6 April, 2009;
originally announced April 2009.
-
Self-Diffusion and Collective Diffusion of Charged colloids Studied by Dynamic Light Scattering
Authors:
Jacqueline Appell,
Grégoire Porte,
Eric Buhler
Abstract:
A microemulsion of decane droplets stabilized by a non-ionic surfactant film is progressively charged by substitution of a non-ionic surfactant molecule by a cationic surfactant. We check that the microemulsion droplets remain identical within the explored range of volume fraction (0.02 to 0.18) and of the number of charge per droplets (0 to 40) . We probe the dynamics of these microemulsions by…
▽ More
A microemulsion of decane droplets stabilized by a non-ionic surfactant film is progressively charged by substitution of a non-ionic surfactant molecule by a cationic surfactant. We check that the microemulsion droplets remain identical within the explored range of volume fraction (0.02 to 0.18) and of the number of charge per droplets (0 to 40) . We probe the dynamics of these microemulsions by dynamic light scattering. Despite the similar structure of the uncharged and charged microemulsions the dynamics are very different . In the neutral microemulsion the fluctuations of polarization relax, as is well known, via the collective diffusion of the droplets. In the charged microemulsions, two modes of relaxation are observed. The fast one is ascribed classically to the collective diffusion of the charged droplets coupled to the diffusion of the counterions. The slow one has, to our knowledge, not been observed previously neither in similar microemulsions nor in charged spherical colloids. We show that the slow mode is also diffusive and suggest that its possible origine is the relaxation of local charge fluctuations via local exchange of droplets bearing different number of charges . The diffusion coefficient associated with this mode is then the self diffusion coefficient of the droplets.
△ Less
Submitted 24 June, 2005;
originally announced June 2005.
