Bin Yu

Predictions based on Machine Learning and Statistics often rely on maximal likelihood, which can trap LLMs in biases inherited from the majority opinions of training data. This problem has been substantially addressed through conditional statistics, allowing for mitigation of biases and preventing the cancellation of minority perspectives. https://lnkd.in/gMXKqwyB

7

Sharing the latest publication from our group: R. Chandra and J. Simmons, "Bayesian neural networks via MCMC: a Python-based tutorial," in IEEE Access, doi: 10.1109/ACCESS.2024.3401234. https://lnkd.in/gS5UfXEp Abstract:  Bayesian inference provides a methodology for parameter estimation and uncertainty quantification in machine learning and deep learning methods. Variational inference and Markov Chain Monte-Carlo (MCMC) sampling methods are used to implement Bayesian inference. In the past three decades, MCMC sampling methods have faced some challenges in being adapted to larger models (such as in deep learning) and big data problems. Advanced proposal distributions that incorporate gradients, such as a Langevin proposal distribution, provide a means to address some of the limitations of MCMC sampling for Bayesian neural networks. Furthermore, MCMC methods have typically been constrained to statisticians and currently not well-known among deep learning researchers. We present a tutorial for MCMC methods that covers simple Bayesian linear and logistic models, and Bayesian neural networks. The aim of this tutorial is to bridge the gap between theory and implementation via coding, given a general sparsity of libraries and tutorials to this end. This tutorial provides code in Python with data and instructions that enable their use and extension. We provide results for some benchmark problems showing the strengths and weaknesses of implementing the respective Bayesian models via MCMC. We highlight the challenges in sampling multi-modal posterior distributions for the case of Bayesian neural networks and the need for further improvement of convergence diagnosis methods. -- Thanks Joshua Simmons ARC Training Centre in Data Analytics for Resources & Environments (DARE) Transitional Artificial Intelligence Research Group

9

2 Comments

Understanding how to perform operations on tensors is crucial due to their fundamental nature

3

New GAI Textbook Announcement (The textbook is being used by Stanford CS372: AI for precision medicine and psychological disorders) The past two years have been a significant period in the field of AI, during which we have observed substantial strides in generative AI technologies. Having dedicated over 5,000 hours to research and analysis, I have compiled this book to share a collection of findings and hypotheses regarding Large Language Models (LLMs)—detailing their operational mechanisms, and strategies to mitigate biases and reduce instances of hallucination. The central premise is to enhance context awareness and define human objectives with greater precision. By doing so, LLMs can more effectively mirror human linguistic behaviors, utilizing context-specific linguistic features to fulfill intended goals. SocraSynth: Socratic Synthesis with Multiple Large Language Models. https://lnkd.in/giC4474p

25

Fine tuning gpt2 on indic-gpt dataset I ran one epoch of fine tuning on an arbitrary dataset of Indic-gpt and managed to train it for language detection with Colab.Have a look: https://lnkd.in/dZJUY-UV #indic-gpt #fine tuning #language detection

10

New Berkeley on understanding large language model: foundations and safety. Lectures on YouTube and covers below topics. https://lnkd.in/g5ukTUV2

9

Researchers at Oxford Presented Policy-Guided Diffusion: A Machine Learning Method for Controllable Generation of Synthetic Trajectories in Offline Reinforcement Learning RL Quick read: https://lnkd.in/g3DdUCgj Researchers from Oxford University present policy-guided diffusion (PGD) to address the issue of compounding error in offline RL by modeling entire trajectories rather than single-step transitions. PGD trains a diffusion model on the offline dataset to generate synthetic trajectories under the behavior policy. To align these trajectories with the target policy, guidance from the target policy is applied to shift the sampling distribution. This results in a behavior-regularized target distribution, reducing divergence from the behavior policy and limiting generalization error. PGD utilizes a trajectory-level diffusion model trained on an offline dataset to approximate the behavior distribution. Inspired by classifier-guided diffusion, PGD incorporates guidance from the target policy during the denoising process to steer trajectory sampling toward the target distribution. This results in a behavior-regularized target distribution, balancing action likelihoods under both policies. PGD excludes behavior policy guidance, focusing solely on target policy guidance. To control guidance strength, PGD introduces guidance coefficients, allowing for fine-tuning of the regularization level towards the behavior distribution. Also, PGD applies a cosine guidance schedule and stabilization techniques to enhance guidance stability and reduce dynamic error. Paper: https://lnkd.in/g3JZDBAP #artificialintelligence University of Oxford #ai

22

Loved the talk. I know category theory but never saw a software implementation side-by-side comparison as he does here using Python and Topos Institute's AlgebraicJulia. Highly recommend it to those that want to think about software engineering in a different way. His use of the triangle graph definitely reminds me of my current implementation of algebraic evaluation in the ntqr Python package. I know the implementation is brittle (coded for exactly three noisy classifiers) and getting the pieces to work has had its issues. I'm definitely going to think about how to do a general n-ensemble implementation using AlgebraicJulia.

11

1 Comment

Dear all, Excited to share our latest publication on Distributed Generative Adversarial Networks! Check out "DRD-GAN: A Novel Distributed Conditional Wasserstein Deep Convolutional Relativistic Discriminator GAN with Improved Convergence" in the ACM Transactions on Probabilistic Machine Learning (2024). Find the publication here: https://lnkd.in/eebkDqcS #GAN #DistributedGAN #FederatedLearning #MachineLearning #Publication #ACM #ResearchPublication

14

1 Comment

Join us for a free talk on "Debiased Inference on Functionals of Inverse Problems and Applications to Long-Term Causal Inference" with Prof. Nathan Kallus, of Cornell University on May 29. For more info & to register visit: https://uoft.me/awk In the presence of endogeneity, instruments and negative controls can still give us a view onto causal effects, but only indirectly, e.g., as a function whose residuals are orthogonal to instruments. Without imposing (unrealistic) parametric restrictions, these inverse problems are generally ill posed, making it difficult to reliably learn a solution from data. Prof. Kallus will discuss how to make reliable inferences on linear functionals of (nonparametric) solutions to these inverse problems, such as average effects. This talk is part of the DSI Causal Inference Emergent Data Science Program. "Debiased Inference on Functionals of Inverse Problems and Applications to Long-Term Causal Inference" 🎤 Nathan Kallus, of Cornell University 📅 May 29, 2024 🕛 12:00-2:30 🏢 Data Sciences Institute, 10th floor, 700 University Avenue 🔗 For more info and to register (free) visit: https://uoft.me/awk 🥙 Includes lunch #Inference #CausalInference #DataScience #EmergentDataScience

13

The Human-or-Machine Issue In the June 2024 issue of Communications of the ACM, David Harel reflects on Alan Turing's "imitation game" and introduces the broader "human-or-machine" (H-or-M) issue. As machines increasingly mimic humans, Harel explores new social, psychological, and technical questions. Will humans care whether their interacting agent is human or machine? How will behaviors and emotional responses differ in human-machine interactions compared to human-human ones? #HumanComputerInteraction #ImitationGame #Interactions #HumanBehavior

3

🚨ALERT: Innovation in artificial neural networks🚨 Published earlier today: “SWAG: A Novel Neural Network Architecture Leveraging Polynomial Activation Functions for Enhanced Deep Learning Efficiency” Kudos to Dr. Saeid Safaei, my last doctoral student at the The University of Georgia. This work is significant because it breaks a dogma in artificial neural network (ANN) literature. In 1993, Leshno et al. reported that an ANN “… can approximate any continuous function to any degree of accuracy if and only if the network's activation function is not a polynomial”. This myth became legend, the legend became dogma. We showed that not only it is possible to use polynomials… it is desirable. Our method with simple architectures outperforms traditional ANNs with complex architectures. It could have profound implications to decrease energy demand for AI through the development of photonic chips. https://lnkd.in/gQAtcNEa

23

8 Comments

Journal Impact Factor JCR-WoS: 2.0 (2023) Q1 CiteScore 2023: 3.1 Q1

9

Hallucination of Multimodal LLMs This new paper presents a comprehensive overview of hallucination in multimodal large language models. Discusses recent advances in detection, evaluation, and mitigation strategies for hallucination. It also summarizes causes, evaluation benchmarks, metrics, challenges, and much more. Paper: https://lnkd.in/gFu2fyRg --- Tracked by 55K+ researchers and developers already, you can also follow my weekly summary of the top AI and LLM papers here: https://lnkd.in/e6ajg945

193

2 Comments

Great step towards #trustworthy #LLMs, which are crucial for LLMs adoption at scale, specially when critical decisions are required to be made (#Finance domains…). Plenty of direct applications e.g., LLMs inputs attribution for #hallucinations detection, #FactChecker !! Even broader: think LLMs outputs now directly attributing LLMs inputs. Great work, Sanjay Kariyappa and team. LLMs inputs and outputs nicely connected with #XAI. #TrustworthyAI

27

I recently attended an enlightening webinar on "Physics-informed Machine Learning (Ф-ML) Meets Engineering," presented by Tobias Nagel, featuring insights from Andrea Pizzoferrato and Zack Xuereb Conti from The Alan Turing Institute. In this session, we explored how combining physics-informed neural networks with the robustness of Kalman filters can revolutionize system identification, especially in noisy and stochastic environments. This innovative approach enhances the accuracy of modeling complex systems, like the double pendulum, by integrating available system knowledge with advanced neural network techniques. Exciting times ahead for machine learning and engineering! #MachineLearning #Engineering #PhysicsInformedML #Webinar #Innovation

16

1 Comment

Phi-3 Vision is now available on Huggingface. Phi series models from MS have smaller sizes but higher accuracy in their size range because the models are trained on a curated high quality dataset. "The model is intended for broad commercial and research use in English. The model provides uses for general purpose AI systems and applications with visual and text input capabilities which require: >> memory/compute constrained environments; >> latency bound scenarios; >> general image understanding; >> OCR; >> chart and table understanding. Our model is designed to accelerate research on efficient language and multimodal models, for use as a building block for generative AI powered features." Mithilesh Thakkar

9