Showing 101–150 of 1,050 results for author: Hong, J

Monostatic imaging of an extended target with MCMC sampling

Authors: Jiho Hong, Sangwoo Kang, Mikyoung Lim

Abstract: We consider the imaging of a planar extended target from far-field data under a monostatic measurement configuration, in which the data is measured by a single moving transducer, as frequently encountered in practical application. In this paper, we develop a Bayesian approach to recover the shape of the extended target with MCMC sampling, where a new shape basis selection is proposed based on the… ▽ More We consider the imaging of a planar extended target from far-field data under a monostatic measurement configuration, in which the data is measured by a single moving transducer, as frequently encountered in practical application. In this paper, we develop a Bayesian approach to recover the shape of the extended target with MCMC sampling, where a new shape basis selection is proposed based on the shape derivative analysis for the measurement data. In order to optimize the center and radius of the initial disk, we use the monostatic sampling method for the center and the explicit scattered field expression for disks for the radius. Numerical simulations are presented to validate the proposed method. △ Less

Submitted 8 December, 2023; originally announced January 2024.

Comments: 17 pages, 8 figures

Stochastic modelling of the instantaneous velocity profile in rough-wall turbulent boundary layers

Authors: Roozbeh Ehsani, Michael Heisel, Jiaqi Li, Vaughan Voller, Jiarong Hong, Michele Guala

Abstract: The statistical properties of Uniform Momentum Zones (UMZs) are extracted from laboratory and field measurements in rough wall turbulent boundary layers to formulate a set of stochastic models for the simulation of instantaneous velocity profiles. A spatio-temporally resolved velocity dataset, covering a field of view of $8 \times 9$ m$^2$, was obtained in the atmospheric surface layer using super… ▽ More The statistical properties of Uniform Momentum Zones (UMZs) are extracted from laboratory and field measurements in rough wall turbulent boundary layers to formulate a set of stochastic models for the simulation of instantaneous velocity profiles. A spatio-temporally resolved velocity dataset, covering a field of view of $8 \times 9$ m$^2$, was obtained in the atmospheric surface layer using super-large-scale particle image velocimetry (SLPIV), as part of the Grand-scale Atmospheric Imaging Apparatus (GAIA). Wind tunnel data from a previous study are included for comparison \citep{heisel2020mixing}. The probability density function of UMZ attributes such as their thickness, modal velocity, and averaged vertical velocity are built at varying elevations and modeled using log-normal and Gaussian distributions. Inverse transform sampling of the distributions is used to generate synthetic step-like velocity profiles that are spatially and temporally uncorrelated. Results show that in the wide range of wall-normal distances and $Re_τ$ up to $ \sim O(10^6)$ investigated here, shear velocity scaling is manifested in the velocity jump across shear interfaces between adjacent UMZs, and attached eddy behavior is observed in the linear proportionality between UMZ thickness and their wall normal location. These very same characteristics are recovered in the generated instantaneous profiles, using both a fully stochastic and a data-driven hybrid stochastic models, which address, in different ways, the coupling between modal velocities and UMZ thickness. Our method provides a stochastic approach for generating an ensemble of instantaneous velocity profiles, consistent with the structural organization of UMZs, where the ensemble reproduces the logarithmic mean velocity profile and recovers significant portions of the Reynolds stresses and thus of the streamwise and vertical velocity variability. △ Less

Submitted 11 January, 2024; originally announced January 2024.

Journal ref: Journal of Fluid Mechanics , Volume 979 , 25 January 2024 , A12

A Survey of Designs for Combined 2D+3D Visual Representations

Authors: Jiayi Hong, Rostyslav Hnatyshyn, Ebrar A. D. Santos, Ross Maciejewski, Tobias Isenberg

Abstract: We examine visual representations of data that make use of combinations of both 2D and 3D data mappings. Combining 2D and 3D representations is a common technique that allows viewers to understand multiple facets of the data with which they are interacting. While 3D representations focus on the spatial character of the data or the dedicated 3D data mapping, 2D representations often show abstract d… ▽ More We examine visual representations of data that make use of combinations of both 2D and 3D data mappings. Combining 2D and 3D representations is a common technique that allows viewers to understand multiple facets of the data with which they are interacting. While 3D representations focus on the spatial character of the data or the dedicated 3D data mapping, 2D representations often show abstract data properties and take advantage of the unique benefits of mapping to a plane. Many systems have used unique combinations of both types of data mappings effectively. Yet there are no systematic reviews of the methods in linking 2D and 3D representations. We systematically survey the relationships between 2D and 3D visual representations in major visualization publications -- IEEE VIS, IEEE TVCG, and EuroVis -- from 2012 to 2022. We closely examined 105 papers where 2D and 3D representations are connected visually, interactively, or through animation. These approaches are designed based on their visual environment, the relationships between their visual representations, and their possible layouts. Through our analysis, we introduce a design space as well as provide design guidelines for effectively linking 2D and 3D visual representations. △ Less

Submitted 12 January, 2024; v1 submitted 8 January, 2024; originally announced January 2024.

Journal ref: IEEE Transactions on Visualization and Computer Graphics 2024

Flexomagnetoelectric effect in Sr2IrO4 thin films

Authors: Xin Liu, Ting Hu, Yujun Zhang, Xueli Xu, Biao Wu, Zongwei Ma, Peng Lv, Yuelin Zhang, Shih-Wen Huang, Jialu Wu, Jing Ma, Jiawang Hong, Zhigao Sheng, Chenglong Jia, Erjun Kan, Ce-Wen Nan, Jinxing Zhang

Abstract: Symmetry engineering is explicitly effective to manipulate and even create phases and orderings in strongly correlated materials. Flexural stress is universally practical to break the space-inversion or time-reversal symmetry. Here, by introducing strain gradient in a centrosymmetric antiferromagnet Sr2IrO4, the space-inversion symmetry is broken accompanying a non-equivalent O p-Ir d orbital hybr… ▽ More Symmetry engineering is explicitly effective to manipulate and even create phases and orderings in strongly correlated materials. Flexural stress is universally practical to break the space-inversion or time-reversal symmetry. Here, by introducing strain gradient in a centrosymmetric antiferromagnet Sr2IrO4, the space-inversion symmetry is broken accompanying a non-equivalent O p-Ir d orbital hybridization along z axis. Thus, emergent polar phase and out-of-plane magnetic moment have been simultaneously observed in these asymmetric Sr2IrO4 thin films, which both are absent in its ground state. Furthermore, upon the application of magnetic field, such polarization can be controlled by modifying the occupied d orbitals through spin-orbit interaction, giving rise to a flexomagnetoelectric effect. This work provides a general strategy to artificially design multiple symmetries and ferroic orderings in strongly correlated systems. △ Less

Submitted 9 January, 2024; originally announced January 2024.

GTA: Guided Transfer of Spatial Attention from Object-Centric Representations

Authors: SeokHyun Seo, Jinwoo Hong, JungWoo Chae, Kyungyul Kim, Sangheum Hwang

Abstract: Utilizing well-trained representations in transfer learning often results in superior performance and faster convergence compared to training from scratch. However, even if such good representations are transferred, a model can easily overfit the limited training dataset and lose the valuable properties of the transferred representations. This phenomenon is more severe in ViT due to its low induct… ▽ More Utilizing well-trained representations in transfer learning often results in superior performance and faster convergence compared to training from scratch. However, even if such good representations are transferred, a model can easily overfit the limited training dataset and lose the valuable properties of the transferred representations. This phenomenon is more severe in ViT due to its low inductive bias. Through experimental analysis using attention maps in ViT, we observe that the rich representations deteriorate when trained on a small dataset. Motivated by this finding, we propose a novel and simple regularization method for ViT called Guided Transfer of spatial Attention (GTA). Our proposed method regularizes the self-attention maps between the source and target models. A target model can fully exploit the knowledge related to object localization properties through this explicit regularization. Our experimental results show that the proposed GTA consistently improves the accuracy across five benchmark datasets especially when the number of training data is small. △ Less

Submitted 5 January, 2024; originally announced January 2024.

Randomly Weighted Neuromodulation in Neural Networks Facilitates Learning of Manifolds Common Across Tasks

Authors: Jinyung Hong, Theodore P. Pavlic

Abstract: Geometric Sensitive Hashing functions, a family of Local Sensitive Hashing functions, are neural network models that learn class-specific manifold geometry in supervised learning. However, given a set of supervised learning tasks, understanding the manifold geometries that can represent each task and the kinds of relationships between the tasks based on them has received little attention. We explo… ▽ More Geometric Sensitive Hashing functions, a family of Local Sensitive Hashing functions, are neural network models that learn class-specific manifold geometry in supervised learning. However, given a set of supervised learning tasks, understanding the manifold geometries that can represent each task and the kinds of relationships between the tasks based on them has received little attention. We explore a formalization of this question by considering a generative process where each task is associated with a high-dimensional manifold, which can be done in brain-like models with neuromodulatory systems. Following this formulation, we define \emph{Task-specific Geometric Sensitive Hashing~(T-GSH)} and show that a randomly weighted neural network with a neuromodulation system can realize this function. △ Less

Submitted 17 November, 2023; originally announced January 2024.

Comments: 10 pages, 7 figures, 1 table, Appear in NeurIPS 2023

A Split-and-Privatize Framework for Large Language Model Fine-Tuning

Authors: Xicong Shen, Yang Liu, Huiqi Liu, Jue Hong, Bing Duan, Zirui Huang, Yunlong Mao, Ye Wu, Di Wu

Abstract: Fine-tuning is a prominent technique to adapt a pre-trained language model to downstream scenarios. In parameter-efficient fine-tuning, only a small subset of modules are trained over the downstream datasets, while leaving the rest of the pre-trained model frozen to save computation resources. In recent years, a popular productization form arises as Model-as-a-Service (MaaS), in which vendors prov… ▽ More Fine-tuning is a prominent technique to adapt a pre-trained language model to downstream scenarios. In parameter-efficient fine-tuning, only a small subset of modules are trained over the downstream datasets, while leaving the rest of the pre-trained model frozen to save computation resources. In recent years, a popular productization form arises as Model-as-a-Service (MaaS), in which vendors provide abundant pre-trained language models, server resources and core functions, and customers can fine-tune, deploy and invoke their customized model by accessing the one-stop MaaS with their own private dataset. In this paper, we identify the model and data privacy leakage risks in MaaS fine-tuning, and propose a Split-and-Privatize (SAP) framework, which manage to mitigate the privacy issues by adapting the existing split learning architecture. The proposed SAP framework is sufficiently investigated by experiments, and the results indicate that it can enhance the empirical privacy by 62% at the cost of 1% model performance degradation on the Stanford Sentiment Treebank dataset. △ Less

Submitted 24 December, 2023; originally announced December 2023.

Towards Better Visualizing the Decision Basis of Networks via Unfold and Conquer Attribution Guidance

Authors: Jung-Ho Hong, Woo-Jeoung Nam, Kyu-Sung Jeon, Seong-Whan Lee

Abstract: Revealing the transparency of Deep Neural Networks (DNNs) has been widely studied to describe the decision mechanisms of network inner structures. In this paper, we propose a novel post-hoc framework, Unfold and Conquer Attribution Guidance (UCAG), which enhances the explainability of the network decision by spatially scrutinizing the input features with respect to the model confidence. Addressing… ▽ More Revealing the transparency of Deep Neural Networks (DNNs) has been widely studied to describe the decision mechanisms of network inner structures. In this paper, we propose a novel post-hoc framework, Unfold and Conquer Attribution Guidance (UCAG), which enhances the explainability of the network decision by spatially scrutinizing the input features with respect to the model confidence. Addressing the phenomenon of missing detailed descriptions, UCAG sequentially complies with the confidence of slices of the image, leading to providing an abundant and clear interpretation. Therefore, it is possible to enhance the representation ability of explanation by preserving the detailed descriptions of assistant input features, which are commonly overwhelmed by the main meaningful regions. We conduct numerous evaluations to validate the performance in several metrics: i) deletion and insertion, ii) (energy-based) pointing games, and iii) positive and negative density maps. Experimental results, including qualitative comparisons, demonstrate that our method outperforms the existing methods with the nature of clear and detailed explanations and applicability. △ Less

Submitted 20 December, 2023; originally announced December 2023.

Comments: 9 pages, 5 figures, Accepted paper in AAAI Conference on Artificial Intelligence (AAAI), 2023

Learning Subject-Aware Cropping by Outpainting Professional Photos

Authors: James Hong, Lu Yuan, Michaël Gharbi, Matthew Fisher, Kayvon Fatahalian

Abstract: How to frame (or crop) a photo often depends on the image subject and its context; e.g., a human portrait. Recent works have defined the subject-aware image cropping task as a nuanced and practical version of image cropping. We propose a weakly-supervised approach (GenCrop) to learn what makes a high-quality, subject-aware crop from professional stock images. Unlike supervised prior work, GenCrop… ▽ More How to frame (or crop) a photo often depends on the image subject and its context; e.g., a human portrait. Recent works have defined the subject-aware image cropping task as a nuanced and practical version of image cropping. We propose a weakly-supervised approach (GenCrop) to learn what makes a high-quality, subject-aware crop from professional stock images. Unlike supervised prior work, GenCrop requires no new manual annotations beyond the existing stock image collection. The key challenge in learning from this data, however, is that the images are already cropped and we do not know what regions were removed. Our insight is to combine a library of stock images with a modern, pre-trained text-to-image diffusion model. The stock image collection provides diversity and its images serve as pseudo-labels for a good crop, while the text-image diffusion model is used to out-paint (i.e., outward inpainting) realistic uncropped images. Using this procedure, we are able to automatically generate a large dataset of cropped-uncropped training pairs to train a cropping model. Despite being weakly-supervised, GenCrop is competitive with state-of-the-art supervised methods and significantly better than comparable weakly-supervised baselines on quantitative and qualitative evaluation metrics. △ Less

Submitted 4 April, 2024; v1 submitted 19 December, 2023; originally announced December 2023.

Comments: AAAI 24. Extended version with supplemental materials

Networking for the Metaverse: The Standardization Landscape

Authors: Cedric Westphal, Jungha Hong, Shin-Gak Kang, Leonardo Chiariglione, Tianji Jiang

Abstract: New applications are being supported by current and future networks. In particular, it is expected that Metaverse applications will be deployed in the near future, as 5G and 6G network provide sufficient bandwidth and sufficiently low latency to provide a satisfying end-user experience. However, networks still need to evolve to better support this type of application. We present here a basic taxon… ▽ More New applications are being supported by current and future networks. In particular, it is expected that Metaverse applications will be deployed in the near future, as 5G and 6G network provide sufficient bandwidth and sufficiently low latency to provide a satisfying end-user experience. However, networks still need to evolve to better support this type of application. We present here a basic taxonomy of the metaverse, which allows to identify some of the networking requirements for such an application; we also provide an overview of the current state of balthe standardization efforts in different standardization organizations, including ITU-T, 3GPP, IETF and MPAI. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Comments: To appear in ITU Journal on Future and Evolving Technologies J-FET December 2023

A compendium of logarithmic corrections in AdS/CFT

Authors: Nikolay Bobev, Marina David, Junho Hong, Valentin Reys, Xuao Zhang

Abstract: We study the logarithmic corrections to various CFT partition functions in the context of the AdS$_4$/CFT$_3$ correspondence for theories arising on the worldvolume of M2-branes. We utilize four-dimensional gauged supergravity and heat kernel methods and present general expressions for the logarithmic corrections to the gravitational on-shell action and black hole entropy for a number of different… ▽ More We study the logarithmic corrections to various CFT partition functions in the context of the AdS$_4$/CFT$_3$ correspondence for theories arising on the worldvolume of M2-branes. We utilize four-dimensional gauged supergravity and heat kernel methods and present general expressions for the logarithmic corrections to the gravitational on-shell action and black hole entropy for a number of different supergravity backgrounds. We outline several subtle features of these calculations and contrast them with a similar analysis of logarithmic corrections performed directly in the eleven-dimensional uplift of a given four-dimensional supergravity background. We find results consistent with AdS/CFT provided that the infinite sum over KK modes on the internal space is regularized in a specific manner. This analysis leads to an explicit expression for the logarithmic correction to the Bekenstein-Hawking entropy of large Kerr-Newmann and Reissner-Nordström black holes in AdS$_4$. Our results also have important implications for effective field theory coupled to gravity in AdS$_4$ and for the existence of scale-separated AdS$_4$ vacua in string theory, which come in the form of new constraints on the field content and mass spectrum of matter fields. △ Less

Submitted 5 April, 2024; v1 submitted 14 December, 2023; originally announced December 2023.

Comments: v1: 87 pages, 10 tables; v2: Seeley-de Witt coefficients in the presence of non-minimal couplings dictated by supersymmetry are newly addressed in section 5 and following places

Hilbert Coefficients and Sally Modules: A Survey of Vasconcelos' Contributions

Authors: Jooyoun Hong, Susan Morey

Abstract: This paper surveys and summarizes Wolmer Vasconcelos' results surrounding multiplicities, Hilbert coefficients, and their extensions. We particularly focus on Vasconcelos' results regarding multiplicities and Chern coefficients, and other invariants which they bound. The Sally module is an important instrument introduced by Vasconcelos for this study, which naturally relates Hilbert coefficients t… ▽ More This paper surveys and summarizes Wolmer Vasconcelos' results surrounding multiplicities, Hilbert coefficients, and their extensions. We particularly focus on Vasconcelos' results regarding multiplicities and Chern coefficients, and other invariants which they bound. The Sally module is an important instrument introduced by Vasconcelos for this study, which naturally relates Hilbert coefficients to reduction numbers. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Comments: 30 pages

MSC Class: 13-02 (Primary) 13D40; 13H10; 13H15 (Secondary)

Neutral Editing Framework for Diffusion-based Video Editing

Authors: Sunjae Yoon, Gwanhyeong Koo, Ji Woo Hong, Chang D. Yoo

Abstract: Text-conditioned image editing has succeeded in various types of editing based on a diffusion framework. Unfortunately, this success did not carry over to a video, which continues to be challenging. Existing video editing systems are still limited to rigid-type editing such as style transfer and object overlay. To this end, this paper proposes Neutral Editing (NeuEdit) framework to enable complex… ▽ More Text-conditioned image editing has succeeded in various types of editing based on a diffusion framework. Unfortunately, this success did not carry over to a video, which continues to be challenging. Existing video editing systems are still limited to rigid-type editing such as style transfer and object overlay. To this end, this paper proposes Neutral Editing (NeuEdit) framework to enable complex non-rigid editing by changing the motion of a person/object in a video, which has never been attempted before. NeuEdit introduces a concept of `neutralization' that enhances a tuning-editing process of diffusion-based editing systems in a model-agnostic manner by leveraging input video and text without any other auxiliary aids (e.g., visual masks, video captions). Extensive experiments on numerous videos demonstrate adaptability and effectiveness of the NeuEdit framework. The website of our work is available here: https://neuedit.github.io △ Less

Submitted 10 December, 2023; originally announced December 2023.

Comments: 18 pages, 14 figures

High Absorptivity Nanotextured Powders for Additive Manufacturing

Authors: Ottman A. Tertuliano, Philip J. DePond, Andrew C. Lee, Jiho Hong, David Doan, Luc Capaldi, Mark Brongersma, X. Wendy Gu, Manyalibo J. Matthews, Wei Cai, Adrian J. Lew

Abstract: The widespread application of metal additive manufacturing (AM) is limited by the ability to control the complex interactions between the energy source and the feedstock material. Here we develop a generalizable process to introduce nanoscale grooves to the surface of metal powders which increases the powder absorptivity by up to 70% during laser powder bed fusion. Absorptivity enhancements in cop… ▽ More The widespread application of metal additive manufacturing (AM) is limited by the ability to control the complex interactions between the energy source and the feedstock material. Here we develop a generalizable process to introduce nanoscale grooves to the surface of metal powders which increases the powder absorptivity by up to 70% during laser powder bed fusion. Absorptivity enhancements in copper, copper-silver, and tungsten enables energy efficient manufacturing, with printing of pure copper at relative densities up to 92% using laser energy densities as low as 82 J/mm^3. Simulations show the enhanced powder absorptivity results from plasmon-enabled light concentration in nanoscale grooves combined with multiple scattering events. The approach taken here demonstrates a general method to enhance the absorptivity and printability of reflective and refractory metal powders by changing the surface morphology of the feedstock without altering its composition. △ Less

Submitted 8 December, 2023; originally announced December 2023.

FedGeo: Privacy-Preserving User Next Location Prediction with Federated Learning

Authors: Chung Park, Taekyoon Choi, Taesan Kim, Mincheol Cho, Junui Hong, Minsung Choi, Jaegul Choo

Abstract: A User Next Location Prediction (UNLP) task, which predicts the next location that a user will move to given his/her trajectory, is an indispensable task for a wide range of applications. Previous studies using large-scale trajectory datasets in a single server have achieved remarkable performance in UNLP task. However, in real-world applications, legal and ethical issues have been raised regardin… ▽ More A User Next Location Prediction (UNLP) task, which predicts the next location that a user will move to given his/her trajectory, is an indispensable task for a wide range of applications. Previous studies using large-scale trajectory datasets in a single server have achieved remarkable performance in UNLP task. However, in real-world applications, legal and ethical issues have been raised regarding privacy concerns leading to restrictions against sharing human trajectory datasets to any other server. In response, Federated Learning (FL) has emerged to address the personal privacy issue by collaboratively training multiple clients (i.e., users) and then aggregating them. While previous studies employed FL for UNLP, they are still unable to achieve reliable performance because of the heterogeneity of clients' mobility. To tackle this problem, we propose the Federated Learning for Geographic Information (FedGeo), a FL framework specialized for UNLP, which alleviates the heterogeneity of clients' mobility and guarantees personal privacy protection. Firstly, we incorporate prior global geographic adjacency information to the local client model, since the spatial correlation between locations is trained partially in each client who has only a heterogeneous subset of the overall trajectories in FL. We also introduce a novel aggregation method that minimizes the gap between client models to solve the problem of client drift caused by differences between client models when learning with their heterogeneous data. Lastly, we probabilistically exclude clients with extremely heterogeneous data from the FL process by focusing on clients who visit relatively diverse locations. We show that FedGeo is superior to other FL methods for model performance in UNLP task. We also validated our model in a real-world application using our own customers' mobile phones and the FL agent system. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Comments: Accepted at 31st ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL 2023)

DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer

Authors: Junyuan Hong, Jiachen T. Wang, Chenhui Zhang, Zhangheng Li, Bo Li, Zhangyang Wang

Abstract: Large Language Models (LLMs) have emerged as dominant tools for various tasks, particularly when tailored for a specific target by prompt tuning. Nevertheless, concerns surrounding data privacy present obstacles due to the tuned prompts' dependency on sensitive private information. A practical solution is to host a local LLM and optimize a soft prompt privately using data. Yet, hosting a local mod… ▽ More Large Language Models (LLMs) have emerged as dominant tools for various tasks, particularly when tailored for a specific target by prompt tuning. Nevertheless, concerns surrounding data privacy present obstacles due to the tuned prompts' dependency on sensitive private information. A practical solution is to host a local LLM and optimize a soft prompt privately using data. Yet, hosting a local model becomes problematic when model ownership is protected. Alternative methods, like sending data to the model's provider for training, intensify these privacy issues facing an untrusted provider. In this paper, we present a novel solution called Differentially-Private Offsite Prompt Tuning (DP-OPT) to address this challenge. Our approach involves tuning a discrete prompt on the client side and then applying it to the desired cloud models. We demonstrate that prompts suggested by LLMs themselves can be transferred without compromising performance significantly. To ensure that the prompts do not leak private information, we introduce the first private prompt generation mechanism, by a differentially-private (DP) ensemble of in-context learning with private demonstrations. With DP-OPT, generating privacy-preserving prompts by Vicuna-7b can yield competitive performance compared to non-private in-context learning on GPT3.5 or local private prompt tuning. Codes are available at https://github.com/VITA-Group/DP-OPT . △ Less

Submitted 17 March, 2024; v1 submitted 26 November, 2023; originally announced December 2023.

Comments: Accepted to ICLR'24 Splotlight (updated version)

Who Leaked the Model? Tracking IP Infringers in Accountable Federated Learning

Authors: Shuyang Yu, Junyuan Hong, Yi Zeng, Fei Wang, Ruoxi Jia, Jiayu Zhou

Abstract: Federated learning (FL) emerges as an effective collaborative learning framework to coordinate data and computation resources from massive and distributed clients in training. Such collaboration results in non-trivial intellectual property (IP) represented by the model parameters that should be protected and shared by the whole party rather than an individual user. Meanwhile, the distributed natur… ▽ More Federated learning (FL) emerges as an effective collaborative learning framework to coordinate data and computation resources from massive and distributed clients in training. Such collaboration results in non-trivial intellectual property (IP) represented by the model parameters that should be protected and shared by the whole party rather than an individual user. Meanwhile, the distributed nature of FL endorses a malicious client the convenience to compromise IP through illegal model leakage to unauthorized third parties. To block such IP leakage, it is essential to make the IP identifiable in the shared model and locate the anonymous infringer who first leaks it. The collective challenges call for \emph{accountable federated learning}, which requires verifiable ownership of the model and is capable of revealing the infringer's identity upon leakage. In this paper, we propose Decodable Unique Watermarking (DUW) for complying with the requirements of accountable FL. Specifically, before a global model is sent to a client in an FL round, DUW encodes a client-unique key into the model by leveraging a backdoor-based watermark injection. To identify the infringer of a leaked model, DUW examines the model and checks if the triggers can be decoded as the corresponding keys. Extensive empirical results show that DUW is highly effective and robust, achieving over $99\%$ watermark success rate for Digits, CIFAR-10, and CIFAR-100 datasets under heterogeneous FL settings, and identifying the IP infringer with $100\%$ accuracy even after common watermark removal attempts. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Deep-learning-driven end-to-end metalens imaging

Authors: Joonhyuk Seo, Jaegang Jo, Joohoon Kim, Joonho Kang, Chanik Kang, Seongwon Moon, Eunji Lee, Jehyeong Hong, Junsuk Rho, Haejun Chung

Abstract: Recent advances in metasurface lenses (metalenses) have shown great potential for opening a new era in compact imaging, photography, light detection and ranging (LiDAR), and virtual reality/augmented reality (VR/AR) applications. However, the fundamental trade-off between broadband focusing efficiency and operating bandwidth limits the performance of broadband metalenses, resulting in chromatic ab… ▽ More Recent advances in metasurface lenses (metalenses) have shown great potential for opening a new era in compact imaging, photography, light detection and ranging (LiDAR), and virtual reality/augmented reality (VR/AR) applications. However, the fundamental trade-off between broadband focusing efficiency and operating bandwidth limits the performance of broadband metalenses, resulting in chromatic aberration, angular aberration, and a relatively low efficiency. In this study, a deep-learning-based image restoration framework is proposed to overcome these limitations and realize end-to-end metalens imaging, thereby achieving aberration-free full-color imaging for mass-produced metalenses with 10-mm diameter. Neural-network-assisted metalens imaging achieved a high resolution comparable to that of the ground truth image. △ Less

Submitted 10 May, 2024; v1 submitted 5 December, 2023; originally announced December 2023.

Comments: 17 pages, 7 figures, 1 table

Visualization and Characterization of Agricultural Sprays Using Machine Learning based Digital Inline Holography

Authors: Shyam Kumar M, Christopher J. Hogan, Steven A. Fredericks, Jiarong Hong

Abstract: Accurate characterization of agricultural sprays is crucial to predict in field performance of liquid applied crop protection products. Here we introduce a robust and efficient machine learning (ML) based Digital In-line Holography (DIH) to accurately characterize the droplet field for a wide range of agricultural spray nozzles. Compared to non-ML methods, our method enhances accuracy, generalizab… ▽ More Accurate characterization of agricultural sprays is crucial to predict in field performance of liquid applied crop protection products. Here we introduce a robust and efficient machine learning (ML) based Digital In-line Holography (DIH) to accurately characterize the droplet field for a wide range of agricultural spray nozzles. Compared to non-ML methods, our method enhances accuracy, generalizability, and processing speed. Our approach employs two neural networks: a modified U-Net to obtain the 3D droplet field from the numerically reconstructed optical field, followed by a VGG16 classifier to reduce false positives from the U-Net prediction. The modified U-Net is trained using holograms generated using a single spray nozzle at three spray locations; center, half-span, and the spray edge to create training data with various number densities and droplet size ranges. VGG16 is trained via the minimum intensity projection of the droplet 3D point spread function. Data augmentation is used to increase the efficiency of classification and make the algorithm generalizable for different measurement settings. The model is validated via NIST traceable glass beads and six agricultural spray nozzles representing various spray characteristics. The results demonstrate a high accuracy rate, with over 90% droplet extraction and less than 5% false positives. Compared to traditional spray measurement techniques, our method offers a significant leap forward in spatial resolution and generalizability. In particular, our method can extract the real cumulative volume distribution of the NIST beads, where the laser diffraction is biased towards droplets moving at slower speeds. Additionally, the ML-based DIH enables the estimation of mass and momentum flux at different locations and the calculation of relative velocities of droplet pairs, which are difficult to obtain via conventional techniques. △ Less

Submitted 13 November, 2023; originally announced December 2023.

Comments: 24 pages, 12 figures

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way

Authors: Kai Lv, Shuo Zhang, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu

Abstract: Large language models (LLMs) are increasingly pivotal in a wide range of natural language processing tasks. Access to pre-trained models, courtesy of the open-source community, has made it possible to adapt these models to specific applications for enhanced performance. However, the substantial resources required for training these models necessitate efficient solutions. This paper introduces CoLL… ▽ More Large language models (LLMs) are increasingly pivotal in a wide range of natural language processing tasks. Access to pre-trained models, courtesy of the open-source community, has made it possible to adapt these models to specific applications for enhanced performance. However, the substantial resources required for training these models necessitate efficient solutions. This paper introduces CoLLiE, an efficient library that facilitates collaborative training of large language models using 3D parallelism, parameter-efficient fine-tuning (PEFT) methods, and optimizers such as Lion, Adan, Sophia, LOMO and AdaLomo. With its modular design and comprehensive functionality, CoLLiE offers a balanced blend of efficiency, ease of use, and customization. CoLLiE has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios. Furthermore, we provide an empirical evaluation of the correlation between model size and GPU memory consumption under different optimization methods, as well as an analysis of the throughput. Lastly, we carry out a comprehensive comparison of various optimizers and PEFT methods within the instruction-tuning context. CoLLiE is available at https://github.com/OpenLMLab/collie. △ Less

Submitted 1 December, 2023; originally announced December 2023.

Comments: To appear at EMNLP 2023 Demo; Code is available at https://github.com/OpenLMLab/collie

LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models

Authors: Marwa Abdulhai, Isadora White, Charlie Snell, Charles Sun, Joey Hong, Yuexiang Zhai, Kelvin Xu, Sergey Levine

Abstract: Large language models (LLMs) provide excellent text-generation capabilities, but standard prompting and generation methods generally do not lead to intentional or goal-directed agents and might necessitate considerable prompt tuning. This becomes particularly apparent in multi-turn conversations: even the best current LLMs rarely ask clarifying questions, engage in explicit information gathering,… ▽ More Large language models (LLMs) provide excellent text-generation capabilities, but standard prompting and generation methods generally do not lead to intentional or goal-directed agents and might necessitate considerable prompt tuning. This becomes particularly apparent in multi-turn conversations: even the best current LLMs rarely ask clarifying questions, engage in explicit information gathering, or take actions now that lead to better decisions after multiple turns. Reinforcement learning has the potential to leverage the powerful modeling capabilities of LLMs, as well as their internal representation of textual interactions, to create capable goal-directed language agents. This can enable intentional and temporally extended interactions, such as with humans, through coordinated persuasion and carefully crafted questions, or in goal-directed play through text games to bring about desired final outcomes. However, enabling this requires the community to develop stable and reliable reinforcement learning algorithms that can effectively train LLMs. Developing such algorithms requires tasks that can gauge progress on algorithm design, provide accessible and reproducible evaluations for multi-turn interactions, and cover a range of task properties and challenges in improving reinforcement learning algorithms. Our paper introduces the LMRL-Gym benchmark for evaluating multi-turn RL for LLMs, together with an open-source research framework containing a basic toolkit for getting started on multi-turn RL with offline value-based and policy-based RL methods. Our benchmark consists of 8 different language tasks, which require multiple rounds of language interaction and cover a range of tasks in open-ended dialogue and text games. △ Less

Submitted 29 November, 2023; originally announced November 2023.

Learning to Simulate: Generative Metamodeling via Quantile Regression

Authors: L. Jeff Hong, Yanxi Hou, Qingkai Zhang, Xiaowei Zhang

Abstract: Stochastic simulation models, while effective in capturing the dynamics of complex systems, are often too slow to run for real-time decision-making. Metamodeling techniques are widely used to learn the relationship between a summary statistic of the outputs (e.g., the mean or quantile) and the inputs of the simulator, so that it can be used in real time. However, this methodology requires the know… ▽ More Stochastic simulation models, while effective in capturing the dynamics of complex systems, are often too slow to run for real-time decision-making. Metamodeling techniques are widely used to learn the relationship between a summary statistic of the outputs (e.g., the mean or quantile) and the inputs of the simulator, so that it can be used in real time. However, this methodology requires the knowledge of an appropriate summary statistic in advance, making it inflexible for many practical situations. In this paper, we propose a new metamodeling concept, called generative metamodeling, which aims to construct a "fast simulator of the simulator". This technique can generate random outputs substantially faster than the original simulation model, while retaining an approximately equal conditional distribution given the same inputs. Once constructed, a generative metamodel can instantaneously generate a large amount of random outputs as soon as the inputs are specified, thereby facilitating the immediate computation of any summary statistic for real-time decision-making. Furthermore, we propose a new algorithm -- quantile-regression-based generative metamodeling (QRGMM) -- and study its convergence and rate of convergence. Extensive numerical experiments are conducted to investigate the empirical performance of QRGMM, compare it with other state-of-the-art generative algorithms, and demonstrate its usefulness in practical real-time decision-making. △ Less

Submitted 29 November, 2023; originally announced November 2023.

Comments: Main body: 36 pages, 7 figures; supplemental material: 12 pages

Three-dimensional internal flow evolution of an evaporating droplet and its role in particle deposition pattern

Authors: Jiaqi Li, Jiarong Hong

Abstract: The internal flow within an evaporating sessile droplet is one of the driving mechanisms that lead to the variety of particle deposition patterns seen in applications such as inkjet printing, surface patterning, and blood stain analysis. Despite decades of research, the causal link between droplet internal flow and particle deposition patterns has not been fully established. In this study, we empl… ▽ More The internal flow within an evaporating sessile droplet is one of the driving mechanisms that lead to the variety of particle deposition patterns seen in applications such as inkjet printing, surface patterning, and blood stain analysis. Despite decades of research, the causal link between droplet internal flow and particle deposition patterns has not been fully established. In this study, we employ a 3D imaging technique based on digital inline holography to quantitatively assess the evolution of internal flow fields and particle migration in three distinct types of wetting droplets: water, sucrose aqueous solution, and SDS aqueous solution droplets, throughout their entire evaporation process. Our imaging reveals the three-stage evolution of the 3D internal flow regimes driven by changes in the relative importance of capillary flow, Marangoni flow, and droplet boundary movement during evaporation, each exhibiting unique dynamics. The migration of particles from their initial locations to deposition can be divided into five categories, with particles depositing either at the contact line or inside the droplet. We observe the changing migration directions of particles due to competing Marangoni and capillary flows during droplet evaporation. We further develop an analytical model that predicts the droplet internal flow and deposition patterns and determines the dependence of the deposition mechanisms of particles on their initial locations and the evolving internal flow field. The model, validated using different types of droplets from our experiment and the literature, can be further expanded to other Newtonian and non-Newtonian droplets, which can potentially serve as a real-time assessment tool for particle deposition in various applications. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Stellar Loci. VII. Photometric Metallicities of 5 Million FGK Stars Based on GALEX GR6+7 AIS and Gaia EDR3

Authors: Xue Lu, Haibo Yuan, Shuai Xu, Ruoyi Zhang, Kai Xiao, Yang Huang, Timothy C. Beers, Jihye Hong

Abstract: We combine photometric data from GALEX GR6+7 AIS and Gaia EDR3 with stellar parameters from the SAGA and PASTEL catalogs to construct high-quality training samples for dwarfs ($\rm 0.4< BP-RP<1.6$) and giants ($\rm 0.6< BP-RP <1.6$). We apply careful reddening corrections using empirical temperature- and extinction-dependent extinction coefficients. Using the two samples, we establish a relationsh… ▽ More We combine photometric data from GALEX GR6+7 AIS and Gaia EDR3 with stellar parameters from the SAGA and PASTEL catalogs to construct high-quality training samples for dwarfs ($\rm 0.4< BP-RP<1.6$) and giants ($\rm 0.6< BP-RP <1.6$). We apply careful reddening corrections using empirical temperature- and extinction-dependent extinction coefficients. Using the two samples, we establish a relationship between stellar loci (NUV$-$BP vs. BP$-$RP colors), metallicity, and $\rm M_G$. For a given BP$-$RP color, a 1 dex change in [Fe/H] corresponds to an approximately 1 magnitude change in NUV$-$BP color for solar-type stars. These relationships are employed to estimate metallicities based on NUV$-$BP, BP$-$RP, and $\rm M_G$. Thanks to the strong metallicity dependence in the GALEX NUV-band, our models enable a typical photometric-metallicity precision of approximately $σ_{\rm [Fe/H]}$ = 0.11 dex for dwarfs and $σ_{\rm [Fe/H]}$ = 0.17 dex for giants, with an effective metallicity range extending down to [Fe/H] $= -3.0$ for dwarfs and [Fe/H] $= -4.0$ for giants. We also find that the NUV-band based photometric-metallicity estimate is not as strongly affected by carbon enhancement as previous photometric techniques. With the Gaia and GALEX data, we have estimated metallicities for about 5 million stars across almost the entire sky, including approximately 4.5 million dwarfs and 0.5 million giants. This work demonstrates the potential of the NUV-band for estimating photometric metallicities, and sets the groundwork for utilizing the NUV data from space telescopes such as the upcoming Chinese Space Station Telescope. △ Less

Submitted 11 January, 2024; v1 submitted 28 November, 2023; originally announced November 2023.

Comments: 24 pages, 19 figures, accepted by ApJS

Exceptional times for the instantaneous propagation of superprocess

Authors: Jieliang Hong, Leonid Mytnik

Abstract: For a Dawson-Watanabe superprocess $X$ on $\mathbb{R}^d$, it is shown in Perkins (1990) that if the underlying spatial motion belongs to a certain class of Lévy processes that admit jumps, then with probability one the closed support of $X_t$ is the whole space for almost all $t>0$ before extinction, the so-called ``instantaneous propagation'' property. In this paper for superprocesses on… ▽ More For a Dawson-Watanabe superprocess $X$ on $\mathbb{R}^d$, it is shown in Perkins (1990) that if the underlying spatial motion belongs to a certain class of Lévy processes that admit jumps, then with probability one the closed support of $X_t$ is the whole space for almost all $t>0$ before extinction, the so-called ``instantaneous propagation'' property. In this paper for superprocesses on $\mathbb{R}^1$ whose spatial motion is the symmetric stable process of index $α\in (0,2/3)$, we prove that there exist exceptional times at which the support is compact and nonempty. Moreover, we show that the set of exceptional times is dense with full Hausdorff dimension. Besides, we prove that near extinction, the support of the superprocess is concentrated arbitrarily close to the distinction point, thus upgrading the corresponding results in Tribe (1992) from $α\in (0,1/2)$ to $α\in (0,2/3)$, and we further show that the set of such exceptional times also admits a full Hausdorff dimension. △ Less

Submitted 6 December, 2023; v1 submitted 22 November, 2023; originally announced November 2023.

Comments: The latest version for this paper is in arXiv:2207.11705

MSC Class: 60J68; 60G17

Machine Learning based Post Event Analysis for Cybersecurity of Cyber-Physical System

Authors: Kuchan Park, Junho Hong, Wencong Su, HyoJong Lee

Abstract: As Information and Communication Technology (ICT) equipment continues to be integrated into power systems, issues related to cybersecurity are increasingly emerging. Particularly noteworthy is the transition to digital substations, which is shifting operations from traditional hardwired-based systems to communication-based Supervisory Control and Data Acquisition (SCADA) system operations. These c… ▽ More As Information and Communication Technology (ICT) equipment continues to be integrated into power systems, issues related to cybersecurity are increasingly emerging. Particularly noteworthy is the transition to digital substations, which is shifting operations from traditional hardwired-based systems to communication-based Supervisory Control and Data Acquisition (SCADA) system operations. These changes in the power system have increased the vulnerability of the system to cyber-attacks and emphasized its importance. This paper proposes a machine learning (ML) based post event analysis of the power system in order to respond to these cybersecurity issues. An artificial neural network (ANN) and other ML models are trained using transient fault measurements and cyber-attack data on substations. The trained models can successfully distinguish between power system faults and cyber-attacks. Furthermore, the results of the proposed ML-based methods can also identify 10 different fault types and the location where the event occurred. △ Less

Submitted 7 March, 2024; v1 submitted 22 November, 2023; originally announced November 2023.

Comments: Submitted to 2024 IEEE Power and Energy Society General Meeting

Cracking the Code of Negative Transfer: A Cooperative Game Theoretic Approach for Cross-Domain Sequential Recommendation

Authors: Chung Park, Taesan Kim, Taekyoon Choi, Junui Hong, Yelim Yu, Mincheol Cho, Kyunam Lee, Sungil Ryu, Hyungjun Yoon, Minsung Choi, Jaegul Choo

Abstract: This paper investigates Cross-Domain Sequential Recommendation (CDSR), a promising method that uses information from multiple domains (more than three) to generate accurate and diverse recommendations, and takes into account the sequential nature of user interactions. The effectiveness of these systems often depends on the complex interplay among the multiple domains. In this dynamic landscape, th… ▽ More This paper investigates Cross-Domain Sequential Recommendation (CDSR), a promising method that uses information from multiple domains (more than three) to generate accurate and diverse recommendations, and takes into account the sequential nature of user interactions. The effectiveness of these systems often depends on the complex interplay among the multiple domains. In this dynamic landscape, the problem of negative transfer arises, where heterogeneous knowledge between dissimilar domains leads to performance degradation due to differences in user preferences across these domains. As a remedy, we propose a new CDSR framework that addresses the problem of negative transfer by assessing the extent of negative transfer from one domain to another and adaptively assigning low weight values to the corresponding prediction losses. To this end, the amount of negative transfer is estimated by measuring the marginal contribution of each domain to model performance based on a cooperative game theory. In addition, a hierarchical contrastive learning approach that incorporates information from the sequence of coarse-level categories into that of fine-level categories (e.g., item level) when implementing contrastive learning was developed to mitigate negative transfer. Despite the potentially low relevance between domains at the fine-level, there may be higher relevance at the category level due to its generalised and broader preferences. We show that our model is superior to prior works in terms of model performance on two real-world datasets across ten different domains. △ Less

Submitted 22 November, 2023; originally announced November 2023.

Comments: Accepted at 32nd ACM International Conference on Information and Knowledge Management (CIKM 2023)

SDN-Based Dynamic Cybersecurity Framework of IEC-61850 Communications in Smart Grid

Authors: Mansi Girdhar, Junho Hong, Wencong Su, Akila Herath, Chen-Ching Liu

Abstract: In recent years, critical infrastructure and power grids have experienced a series of cyber-attacks, leading to temporary, widespread blackouts of considerable magnitude. Since most substations are unmanned and have limited physical security protection, cyber breaches into power grid substations present a risk. Nowadays, software-defined network (SDN), a popular virtual network technology based on… ▽ More In recent years, critical infrastructure and power grids have experienced a series of cyber-attacks, leading to temporary, widespread blackouts of considerable magnitude. Since most substations are unmanned and have limited physical security protection, cyber breaches into power grid substations present a risk. Nowadays, software-defined network (SDN), a popular virtual network technology based on the OpenFlow protocol is being widely used in the substation automation system. However, the susceptibility of SDN architecture to cyber-attacks has exhibited a notable increase in recent years, as indicated by research findings. This suggests a growing concern regarding the potential for cybersecurity breaches within the SDN framework. In this paper, we propose a hybrid intrusion detection system (IDS)-integrated SDN architecture for detecting and preventing the injection of malicious IEC 61850-based generic object-oriented substation event (GOOSE) messages in a digital substation. Additionally, this program locates the fault's location and, as a form of mitigation, disables a certain port. Furthermore, implementation examples are demonstrated and verified using a hardware-in-the-loop (HIL) testbed that mimics the functioning of a digital substation. △ Less

Submitted 7 March, 2024; v1 submitted 20 November, 2023; originally announced November 2023.

Comments: 5 pages, 6 figures, 1 table, conference paper, supported by DOE (CESER) program

Staffing under Taylor's Law: A Unifying Framework for Bridging Square-root and Linear Safety Rules

Authors: L. Jeff Hong, Weihuan Huang, Jiheng Zhang, Xiaowei Zhang

Abstract: Staffing rules serve as an essential management tool in service industries to attain target service levels. Traditionally, the square-root safety rule, based on the Poisson arrival assumption, has been commonly used. However, empirical findings suggest that arrival processes often exhibit an ``over-dispersion'' phenomenon, in which the variance of the arrival exceeds the mean. In this paper, we de… ▽ More Staffing rules serve as an essential management tool in service industries to attain target service levels. Traditionally, the square-root safety rule, based on the Poisson arrival assumption, has been commonly used. However, empirical findings suggest that arrival processes often exhibit an ``over-dispersion'' phenomenon, in which the variance of the arrival exceeds the mean. In this paper, we develop a new doubly stochastic Poisson process model to capture a significant dispersion scaling law, known as Taylor's law, showing that the variance is a power function of the mean. We further examine how over-dispersion affects staffing, providing a closed-form staffing formula to ensure a desired service level. Interestingly, the additional staffing level beyond the nominal load is a power function of the nominal load, with the power exponent lying between $1/2$ (the square-root safety rule) and $1$ (the linear safety rule), depending on the degree of over-dispersion. Simulation studies and a large-scale call center case study indicate that our staffing rule outperforms classical alternatives. △ Less

Submitted 19 November, 2023; originally announced November 2023.

Comments: 55 pages

Bayesian Neural Networks: A Min-Max Game Framework

Authors: Junping Hong, Ercan Engin Kuruoglu

Abstract: This paper is a preliminary study of the robustness and noise analysis of deep neural networks via a game theory formulation Bayesian Neural Networks (BNN) and the maximal coding rate distortion loss. BNN has been shown to provide some robustness to deep learning, and the minimax method used to be a natural conservative way to assist the Bayesian method. Inspired by the recent closed-loop transcri… ▽ More This paper is a preliminary study of the robustness and noise analysis of deep neural networks via a game theory formulation Bayesian Neural Networks (BNN) and the maximal coding rate distortion loss. BNN has been shown to provide some robustness to deep learning, and the minimax method used to be a natural conservative way to assist the Bayesian method. Inspired by the recent closed-loop transcription neural network, we formulate the BNN via game theory between the deterministic neural network $f$ and the sampling network $f + ξ$ or $f + r*ξ$. Compared with previous BNN, BNN via game theory learns a solution space within a certain gap between the center $f$ and the sampling point $f + r*ξ$, and is a conservative choice with a meaningful prior setting compared with previous BNN. Furthermore, the minimum points between $f$ and $f + r*ξ$ become stable when the subspace dimension is large enough with a well-trained model $f$. With these, the model $f$ can have a high chance of recognizing the out-of-distribution data or noise data in the subspace rather than the prediction level, even if $f$ is in online training after a few iterations of true data. So far, our experiments are limited to MNIST and Fashion MNIST data sets, more experiments with realistic data sets and complicated neural network models should be implemented to validate the above arguments. △ Less

Submitted 29 May, 2024; v1 submitted 18 November, 2023; originally announced November 2023.

Comments: 6 pages, 8 figures,

Stable Differentiable Causal Discovery

Authors: Achille Nazaret, Justin Hong, Elham Azizi, David Blei

Abstract: Inferring causal relationships as directed acyclic graphs (DAGs) is an important but challenging problem. Differentiable Causal Discovery (DCD) is a promising approach to this problem, framing the search as a continuous optimization. But existing DCD methods are numerically unstable, with poor performance beyond tens of variables. In this paper, we propose Stable Differentiable Causal Discovery (S… ▽ More Inferring causal relationships as directed acyclic graphs (DAGs) is an important but challenging problem. Differentiable Causal Discovery (DCD) is a promising approach to this problem, framing the search as a continuous optimization. But existing DCD methods are numerically unstable, with poor performance beyond tens of variables. In this paper, we propose Stable Differentiable Causal Discovery (SDCD), a new method that improves previous DCD methods in two ways: (1) It employs an alternative constraint for acyclicity; this constraint is more stable, both theoretically and empirically, and fast to compute. (2) It uses a training procedure tailored for sparse causal graphs, which are common in real-world scenarios. We first derive SDCD and prove its stability and correctness. We then evaluate it with both observational and interventional data and on both small-scale and large-scale settings. We find that SDCD outperforms existing methods in both convergence speed and accuracy and can scale to thousands of variables. We provide code at https://github.com/azizilab/sdcd. △ Less

Submitted 27 June, 2024; v1 submitted 16 November, 2023; originally announced November 2023.

Topological and control theoretic properties of Hamilton-Jacobi equations via Lax-Oleinik commutators

Authors: Piermarco Cannarsa, Wei Cheng, Jiahui Hong

Abstract: In the context of weak KAM theory, we discuss the commutators $\{T^-_t\circ T^+_t\}_{t\geqslant0}$ and $\{T^+_t\circ T^-_t\}_{t\geqslant0}$ of Lax-Oleinik operators. We characterize the relation $T^-_t\circ T^+_t=Id$ for both small time and arbitrary time $t$. We show this relation characterizes controllability for evolutionary Hamilton-Jacobi equation. Based on our previous work on the cut locus… ▽ More In the context of weak KAM theory, we discuss the commutators $\{T^-_t\circ T^+_t\}_{t\geqslant0}$ and $\{T^+_t\circ T^-_t\}_{t\geqslant0}$ of Lax-Oleinik operators. We characterize the relation $T^-_t\circ T^+_t=Id$ for both small time and arbitrary time $t$. We show this relation characterizes controllability for evolutionary Hamilton-Jacobi equation. Based on our previous work on the cut locus of viscosity solution, we refine our analysis of the cut time function $τ$ in terms of commutators $T^+_t\circ T^-_t-T^+_t\circ T^-_t$ and clarify the structure of the super/sub-level set of the cut time function $τ$. △ Less

Submitted 12 November, 2023; originally announced November 2023.

Joint Design of Coding and Modulation for Digital Over-the-Air Computation

Authors: Xin Xie, Cunqinq Hua, Jianan Hong, Yuejun Wei

Abstract: Due to its high communication efficiency, over-the-air computation (AirComp) has been expected to carry out various computing tasks in the next-generation wireless networks. However, up to now, most applications of AirComp are explored in the analog domain, which limits the capability of AirComp in resisting the complex wireless environment, not to mention to integrate the AirComp technique to the… ▽ More Due to its high communication efficiency, over-the-air computation (AirComp) has been expected to carry out various computing tasks in the next-generation wireless networks. However, up to now, most applications of AirComp are explored in the analog domain, which limits the capability of AirComp in resisting the complex wireless environment, not to mention to integrate the AirComp technique to the existing universal communication standards, most of which are based on the digital system. In this paper, we propose a joint design of channel coding and digital modulation for digital AirComp transmission to attempt to reinforce the foundation for the application of AirComp in the digital system. Specifically, we first propose a non-binary LDPC-based channel coding scheme to enhance the error-correction capability of AirComp. Then, a digital modulation scheme is proposed to achieve the number summation from multiple transmitters via the lattice coding technique. We also provide simulation results to demonstrate the feasibility and the performance of the proposed design. △ Less

Submitted 12 November, 2023; originally announced November 2023.

Comments: This paper has been submitted to IEEE ICC 2024

The Effect of Trust and its Antecedents on Robot Acceptance

Authors: Katrin Fischer, Donggyu Kim, Joo-Wha Hong

Abstract: As social and socially assistive robots are becoming more prevalent in our society, it is beneficial to understand how people form first impressions of them and eventually come to trust and accept them. This paper describes an Amazon Mechanical Turk study (n = 239) that investigated trust and its antecedents trustworthiness and first impressions. Participants evaluated the social robot Pepper's wa… ▽ More As social and socially assistive robots are becoming more prevalent in our society, it is beneficial to understand how people form first impressions of them and eventually come to trust and accept them. This paper describes an Amazon Mechanical Turk study (n = 239) that investigated trust and its antecedents trustworthiness and first impressions. Participants evaluated the social robot Pepper's warmth and competence as well as trustworthiness characteristics ability, benevolence and integrity followed by their trust in and intention to use the robot. Mediation analyses assessed to what degree participants' first impressions affected their willingness to trust and use it. Known constructs from user acceptance and trust research were introduced to explain the pathways in which one perception predicted the next. Results showed that trustworthiness and trust, in serial, mediated the relationship between first impressions and behavioral intention. △ Less

Submitted 11 November, 2023; originally announced November 2023.

Comments: In SCRITA 2023 Workshop Proceedings (arXiv:2311.05401) held in conjunction with 32nd IEEE International Conference on Robot & Human Interactive Communication, 28/08 - 31/08 2023, Busan (Korea)

Report number: SCRITA/2023/03

Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations

Authors: Joey Hong, Sergey Levine, Anca Dragan

Abstract: Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks. However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome. For example, a teacher might try to understand their student's current comprehension level to tailor their instruction accordingly, and… ▽ More Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks. However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome. For example, a teacher might try to understand their student's current comprehension level to tailor their instruction accordingly, and a travel agent might ask questions of their customer to understand their preferences in order to recommend activities they might enjoy. LLMs trained with supervised fine-tuning or "single-step" RL, as with standard RLHF, might struggle which tasks that require such goal-directed behavior, since they are not trained to optimize for overall conversational outcomes after multiple turns of interaction. In this work, we explore a new method for adapting LLMs with RL for such goal-directed dialogue. Our key insight is that, though LLMs might not effectively solve goal-directed dialogue tasks out of the box, they can provide useful data for solving such tasks by simulating suboptimal but human-like behaviors. Given a textual description of a goal-directed dialogue task, we leverage LLMs to sample diverse synthetic rollouts of hypothetical in-domain human-human interactions. Our algorithm then utilizes this dataset with offline reinforcement learning to train an interactive conversational agent that can optimize goal-directed objectives over multiple turns. In effect, the LLM produces examples of possible interactions, and RL then processes these examples to learn to perform more optimal interactions. Empirically, we show that our proposed approach achieves state-of-the-art performance in various goal-directed dialogue tasks that include teaching and preference elicitation. △ Less

Submitted 9 November, 2023; originally announced November 2023.

Comments: 25 pages, 6 figures

ChatGPT and Other Large Language Models for Cybersecurity of Smart Grid Applications

Authors: Aydin Zaboli, Seong Lok Choi, Tai-Jin Song, Junho Hong

Abstract: Cybersecurity breaches targeting electrical substations constitute a significant threat to the integrity of the power grid, necessitating comprehensive defense and mitigation strategies. Any anomaly in information and communication technology (ICT) should be detected for secure communications between devices in digital substations. This paper proposes large language models (LLM), e.g., ChatGPT, fo… ▽ More Cybersecurity breaches targeting electrical substations constitute a significant threat to the integrity of the power grid, necessitating comprehensive defense and mitigation strategies. Any anomaly in information and communication technology (ICT) should be detected for secure communications between devices in digital substations. This paper proposes large language models (LLM), e.g., ChatGPT, for the cybersecurity of IEC 61850-based digital substation communications. Multicast messages such as generic object oriented system event (GOOSE) and sampled value (SV) are used for case studies. The proposed LLM-based cybersecurity framework includes, for the first time, data pre-processing of communication systems and human-in-the-loop (HITL) training (considering the cybersecurity guidelines recommended by humans). The results show a comparative analysis of detected anomaly data carried out based on the performance evaluation metrics for different LLMs. A hardware-in-the-loop (HIL) testbed is used to generate and extract dataset of IEC 61850 communications. △ Less

Submitted 25 February, 2024; v1 submitted 9 November, 2023; originally announced November 2023.

Comments: 5 pages, 2 figures, Accepted, 2024 IEEE Power & Energy Society General Meeting (PESGM), Seattle, WA, USA

The High Energy X-ray Probe (HEX-P): resolving the nature of Sgr A* flares, compact object binaries and diffuse X-ray emission in the Galactic Center and beyond

Authors: Kaya Mori, Gabriele Ponti, Matteo Bachetti, Arash Bodaghee, Jonathan Grindlay, Jaesub Hong, Roman Krivonos, Ekaterina Kuznetsova, Shifra Mandel, Antonio Rodriguez, Giovanni Stel, Shuo Zhang, Tong Bao, Franz Bauer, Maica Clavel, Benjamin Coughenour, Javier A. Garcia, Julian Gerber, Brian Grefenstette, Amruta Jaodand, Bret Lehmer, Kristin Madsen, Melania Nynka, Peter Predehl, Ciro Salcedo , et al. (2 additional authors not shown)

Abstract: HEX-P is a probe-class mission concept that will combine high spatial resolution X-ray imaging ($<10"$ FWHM) and broad spectral coverage (0.2-80 keV) with an effective area far superior to current facilities' (including XMM-Newton and NuSTAR). These capabilities will enable revolutionary new insights into a variety of important astrophysical problems. We present scientific objectives and simulatio… ▽ More HEX-P is a probe-class mission concept that will combine high spatial resolution X-ray imaging ($<10"$ FWHM) and broad spectral coverage (0.2-80 keV) with an effective area far superior to current facilities' (including XMM-Newton and NuSTAR). These capabilities will enable revolutionary new insights into a variety of important astrophysical problems. We present scientific objectives and simulations of HEX-P observations of the Galactic Center (GC) and Bulge. We demonstrate the unique and powerful capabilities of the HEX-P observatory for studying both X-ray point sources and diffuse X-ray emission. HEX-P will be uniquely equipped to explore a variety of major topics in Galactic astrophysics, allowing us to (1) investigate broad-band properties of X-ray flares emitted from the supermassive black hole (BH) at Sgr A* and probe the associated particle acceleration and emission mechanisms; (2) identify hard X-ray sources detected by NuSTAR and determine X-ray point source populations in different regions and luminosity ranges; (3) determine the distribution of compact object binaries in the nuclear star cluster and the composition of the Galactic Ridge X-ray emission; (4) identify X-ray transients and measure fundamental parameters such as BH spin; (5) find hidden pulsars in the GC; (6) search for BH-OB binaries and hard X-ray flares from young stellar objects in young massive clusters; (7) measure white dwarf (WD) masses of magnetic CVs to deepen our understanding of CV evolution and the origin of WD magnetic fields; (8) explore primary particle accelerators in the GC in synergy with future TeV and neutrino observatories; (9) map out cosmic-ray distributions by observing non-thermal X-ray filaments; (10) explore past X-ray outbursts from Sgr A* through X-ray reflection components from giant molecular clouds. △ Less

Submitted 8 November, 2023; originally announced November 2023.

Candidate Members of the VMP/EMP Disk System of the Galaxy from the SkyMapper and SAGES Surveys

Authors: Jihye Hong, Timothy C. Beers, Young Sun Lee, Yang Huang, Yutaka Hirai, Jonathan Cabrera Garcia, Derek Shank, Shuai Xu, Haibo Yuan, Mohammad K. Mardini, Thomas Catapano, Gang Zhao, Zhou Fan, Jie Zheng, Wei Wang, Kefeng Tan, Jingkun Zhao, Chun Li

Abstract: Photometric stellar surveys now cover a large fraction of the sky, probe to fainter magnitudes than large-scale spectroscopic surveys, and are relatively free from the target-selection biases often associated with such studies. Photometric-metallicity estimates that include narrow/medium-band filters can achieve comparable accuracy and precision to existing low-resolution spectroscopic surveys suc… ▽ More Photometric stellar surveys now cover a large fraction of the sky, probe to fainter magnitudes than large-scale spectroscopic surveys, and are relatively free from the target-selection biases often associated with such studies. Photometric-metallicity estimates that include narrow/medium-band filters can achieve comparable accuracy and precision to existing low-resolution spectroscopic surveys such as SDSS/SEGUE and LAMOST. Here we report on an effort to identify likely members of the Galactic disk system among the very metal-poor (VMP; [Fe/H] $\leq$ --2) and extremely metal-poor (EMP; [Fe/H] $\leq$ --3) stars. Our analysis is based on an initial sample of $\sim11.5$ million stars with full space motions selected from the SkyMapper Southern Survey (SMSS) and Stellar Abundance and Galactic Evolution Survey (SAGES). After applying a number of quality cuts to obtain the best available metallicity and dynamical estimates, we analyze a total of $\sim$5.86 million stars in the combined SMSS/SAGES sample. We employ two techniques that, depending on the method, identify between 876 and 1,476 VMP stars (6.9%-11.7% of all VMP stars) and between 40 and 59 EMP stars (12.4%-18.3% of all EMP stars) that appear to be members of the Galactic disk system on highly prograde orbits (v$_φ > 150$ km/s). The total number of candidate VMP/EMP disk-like stars is 1,496, the majority of which have low orbital eccentricities, ecc $\le 0.4$; many have ecc $\le 0.2$. The large fractions of VMP/EMP stars associated with the Milky Way disk system strongly suggest the presence of an early forming ``primordial" disk. △ Less

Submitted 19 June, 2024; v1 submitted 3 November, 2023; originally announced November 2023.

Comments: It is now accepted to the ApJS

Relation Extraction from News Articles (RENA): A Tool for Epidemic Surveillance

Authors: Jaeff Hong, Duong Dung, Danielle Hutchinson, Zubair Akhtar, Rosalie Chen, Rebecca Dawson, Aditya Joshi, Samsung Lim, C Raina MacIntyre, Deepti Gurdasani

Abstract: Relation Extraction from News Articles (RENA) is a browser-based tool designed to extract key entities and their semantic relationships in English language news articles related to infectious diseases. Constructed using the React framework, this system presents users with an elegant and user-friendly interface. It enables users to input a news article and select from a choice of two models to gene… ▽ More Relation Extraction from News Articles (RENA) is a browser-based tool designed to extract key entities and their semantic relationships in English language news articles related to infectious diseases. Constructed using the React framework, this system presents users with an elegant and user-friendly interface. It enables users to input a news article and select from a choice of two models to generate a comprehensive list of relations within the provided text. As a result, RENA allows real-time parsing of news articles to extract key information for epidemic surveillance, contributing to EPIWATCH, an open-source intelligence-based epidemic warning system. △ Less

Submitted 31 October, 2023; originally announced November 2023.

Comments: Under review in AAAI 2024

Mixture-of-Experts for Open Set Domain Adaptation: A Dual-Space Detection Approach

Authors: Zhenbang Du, Jiayu An, Yunlu Tu, Jiahao Hong, Dongrui Wu

Abstract: Open Set Domain Adaptation (OSDA) aims to cope with the distribution and label shifts between the source and target domains simultaneously, performing accurate classification for known classes while identifying unknown class samples in the target domain. Most existing OSDA approaches, depending on the final image feature space of deep models, require manually-tuned thresholds, and may easily miscl… ▽ More Open Set Domain Adaptation (OSDA) aims to cope with the distribution and label shifts between the source and target domains simultaneously, performing accurate classification for known classes while identifying unknown class samples in the target domain. Most existing OSDA approaches, depending on the final image feature space of deep models, require manually-tuned thresholds, and may easily misclassify unknown samples as known classes. Mixture-of-Experts (MoE) could be a remedy. Within a MoE, different experts handle distinct input features, producing unique expert routing patterns for various classes in a routing feature space. As a result, unknown class samples may display different expert routing patterns to known classes. In this paper, we propose Dual-Space Detection, which exploits the inconsistencies between the image feature space and the routing feature space to detect unknown class samples without any threshold. Graph Router is further introduced to better make use of the spatial information among image patches. Experiments on three different datasets validated the effectiveness and superiority of our approach. △ Less

Submitted 3 July, 2024; v1 submitted 1 November, 2023; originally announced November 2023.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Offline RL with Observation Histories: Analyzing and Improving Sample Complexity

Authors: Joey Hong, Anca Dragan, Sergey Levine

Abstract: Offline reinforcement learning (RL) can in principle synthesize more optimal behavior from a dataset consisting only of suboptimal trials. One way that this can happen is by "stitching" together the best parts of otherwise suboptimal trajectories that overlap on similar states, to create new behaviors where each individual state is in-distribution, but the overall returns are higher. However, in m… ▽ More Offline reinforcement learning (RL) can in principle synthesize more optimal behavior from a dataset consisting only of suboptimal trials. One way that this can happen is by "stitching" together the best parts of otherwise suboptimal trajectories that overlap on similar states, to create new behaviors where each individual state is in-distribution, but the overall returns are higher. However, in many interesting and complex applications, such as autonomous navigation and dialogue systems, the state is partially observed. Even worse, the state representation is unknown or not easy to define. In such cases, policies and value functions are often conditioned on observation histories instead of states. In these cases, it is not clear if the same kind of "stitching" is feasible at the level of observation histories, since two different trajectories would always have different histories, and thus "similar states" that might lead to effective stitching cannot be leveraged. Theoretically, we show that standard offline RL algorithms conditioned on observation histories suffer from poor sample complexity, in accordance with the above intuition. We then identify sufficient conditions under which offline RL can still be efficient -- intuitively, it needs to learn a compact representation of history comprising only features relevant for action selection. We introduce a bisimulation loss that captures the extent to which this happens, and propose that offline RL can explicitly optimize this loss to aid worst-case sample complexity. Empirically, we show that across a variety of tasks either our proposed loss improves performance, or the value of this loss is already minimized as a consequence of standard offline RL, indicating that it correlates well with good performance. △ Less

Submitted 31 October, 2023; originally announced October 2023.

Comments: 21 pages, 4 figures

Unmasking Bias in AI: A Systematic Review of Bias Detection and Mitigation Strategies in Electronic Health Record-based Models

Authors: Feng Chen, Liqin Wang, Julie Hong, Jiaqi Jiang, Li Zhou

Abstract: Objectives: Leveraging artificial intelligence (AI) in conjunction with electronic health records (EHRs) holds transformative potential to improve healthcare. Yet, addressing bias in AI, which risks worsening healthcare disparities, cannot be overlooked. This study reviews methods to detect and mitigate diverse forms of bias in AI models developed using EHR data. Methods: We conducted a systematic… ▽ More Objectives: Leveraging artificial intelligence (AI) in conjunction with electronic health records (EHRs) holds transformative potential to improve healthcare. Yet, addressing bias in AI, which risks worsening healthcare disparities, cannot be overlooked. This study reviews methods to detect and mitigate diverse forms of bias in AI models developed using EHR data. Methods: We conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines, analyzing articles from PubMed, Web of Science, and IEEE published between January 1, 2010, and Dec 17, 2023. The review identified key biases, outlined strategies for detecting and mitigating bias throughout the AI model development process, and analyzed metrics for bias assessment. Results: Of the 450 articles retrieved, 20 met our criteria, revealing six major bias types: algorithmic, confounding, implicit, measurement, selection, and temporal. The AI models were primarily developed for predictive tasks in healthcare settings. Four studies concentrated on the detection of implicit and algorithmic biases employing fairness metrics like statistical parity, equal opportunity, and predictive equity. Sixty proposed various strategies for mitigating biases, especially targeting implicit and selection biases. These strategies, evaluated through both performance (e.g., accuracy, AUROC) and fairness metrics, predominantly involved data collection and preprocessing techniques like resampling, reweighting, and transformation. Discussion: This review highlights the varied and evolving nature of strategies to address bias in EHR-based AI models, emphasizing the urgent needs for the establishment of standardized, generalizable, and interpretable methodologies to foster the creation of ethical AI systems that promote fairness and equity in healthcare. △ Less

Submitted 1 July, 2024; v1 submitted 30 October, 2023; originally announced October 2023.

Comments: Published in JAMIA Volume 31, Issue 5, May 2024

Probing dynamics of elliptical vortex rings via direct vorticity measurements with digital inline holography

Authors: Jiaqi Li, Jiarong Hong

Abstract: Investigating vorticity dynamics provides an effective way for understanding the fundamental mechanisms of fluid flows across diverse scales. However, experimental vorticity measurements often suffer from limited spatial and temporal resolution, hindering our capability to probe into small-scale dynamics in various flows, particularly turbulence. In Li et al. (EXIF, 2022, vol. 63, 161), we introdu… ▽ More Investigating vorticity dynamics provides an effective way for understanding the fundamental mechanisms of fluid flows across diverse scales. However, experimental vorticity measurements often suffer from limited spatial and temporal resolution, hindering our capability to probe into small-scale dynamics in various flows, particularly turbulence. In Li et al. (EXIF, 2022, vol. 63, 161), we introduced a novel holographic vorticimetry technique for direct vorticity measurements by tracking the three-dimensional rotations of tracers with internal markers. This study further extends it to investigate the intricate vorticity dynamics during the evolution of elliptical vortex rings with different aspect ratios. Based on the shadowgraph imaging quantifying the axis-switching cycles and vortex ring deformation, holographic vorticimetry is applied to measure the vorticity distribution within the millimeter-size core of elliptical vortex rings during their evolution. Specifically, our method resolves an even vorticity spread near the core center that rapidly decays within a few hundred microns. Additionally, our results reveal the intricate vorticity fluctuations associated with the folding-unfolding behaviors during the vortex ring evolution. These subtle vorticity changes informed by simulations have not been captured by previous experiments due to limited resolution. Furthermore, we find that higher aspect ratios yield larger initial vorticity and vorticity fluctuations but also prompt earlier inception of instabilities, causing vortex core distortion. These opposing effects result in a non-monotonic vorticity evolution trend. Overall, our measurements demonstrate the efficacy of holographic vorticimetry by measuring the intricate vorticity variations in unsteady vortex flows, paving the way for capturing the vorticity dynamics of small-scale turbulence structures. △ Less

Submitted 30 October, 2023; originally announced October 2023.

Leveraging Large Language Models for Enhanced Product Descriptions in eCommerce

Authors: Jianghong Zhou, Bo Liu, Jhalak Nilesh Acharya Yao Hong, Kuang-chih Lee, Musen Wen

Abstract: In the dynamic field of eCommerce, the quality and comprehensiveness of product descriptions are pivotal for enhancing search visibility and customer engagement. Effective product descriptions can address the 'cold start' problem, align with market trends, and ultimately lead to increased click-through rates. Traditional methods for crafting these descriptions often involve significant human effor… ▽ More In the dynamic field of eCommerce, the quality and comprehensiveness of product descriptions are pivotal for enhancing search visibility and customer engagement. Effective product descriptions can address the 'cold start' problem, align with market trends, and ultimately lead to increased click-through rates. Traditional methods for crafting these descriptions often involve significant human effort and may lack both consistency and scalability. This paper introduces a novel methodology for automating product description generation using the LLAMA 2.0 7B language model. We train the model on a dataset of authentic product descriptions from Walmart, one of the largest eCommerce platforms. The model is then fine-tuned for domain-specific language features and eCommerce nuances to enhance its utility in sales and user engagement. We employ multiple evaluation metrics, including NDCG, customer click-through rates, and human assessments, to validate the effectiveness of our approach. Our findings reveal that the system is not only scalable but also significantly reduces the human workload involved in creating product descriptions. This study underscores the considerable potential of large language models like LLAMA 2.0 7B in automating and optimizing various facets of eCommerce platforms, offering significant business impact, including improved search functionality and increased sales. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Comments: 9 pages, 4 figures, EMNLP2023 workshop, The 2023 Conference on Empirical Methods in Natural Language Processing

ACM Class: I.2.7; H.3.3; H.3.5; K.4.4

SoK: Pitfalls in Evaluating Black-Box Attacks

Authors: Fnu Suya, Anshuman Suri, Tingwei Zhang, Jingtao Hong, Yuan Tian, David Evans

Abstract: Numerous works study black-box attacks on image classifiers. However, these works make different assumptions on the adversary's knowledge and current literature lacks a cohesive organization centered around the threat model. To systematize knowledge in this area, we propose a taxonomy over the threat space spanning the axes of feedback granularity, the access of interactive queries, and the qualit… ▽ More Numerous works study black-box attacks on image classifiers. However, these works make different assumptions on the adversary's knowledge and current literature lacks a cohesive organization centered around the threat model. To systematize knowledge in this area, we propose a taxonomy over the threat space spanning the axes of feedback granularity, the access of interactive queries, and the quality and quantity of the auxiliary data available to the attacker. Our new taxonomy provides three key insights. 1) Despite extensive literature, numerous under-explored threat spaces exist, which cannot be trivially solved by adapting techniques from well-explored settings. We demonstrate this by establishing a new state-of-the-art in the less-studied setting of access to top-k confidence scores by adapting techniques from well-explored settings of accessing the complete confidence vector, but show how it still falls short of the more restrictive setting that only obtains the prediction label, highlighting the need for more research. 2) Identification the threat model of different attacks uncovers stronger baselines that challenge prior state-of-the-art claims. We demonstrate this by enhancing an initially weaker baseline (under interactive query access) via surrogate models, effectively overturning claims in the respective paper. 3) Our taxonomy reveals interactions between attacker knowledge that connect well to related areas, such as model inversion and extraction attacks. We discuss how advances in other areas can enable potentially stronger black-box attacks. Finally, we emphasize the need for a more realistic assessment of attack success by factoring in local attack runtime. This approach reveals the potential for certain attacks to achieve notably higher success rates and the need to evaluate attacks in diverse and harder settings, highlighting the need for better selection criteria. △ Less

Submitted 14 February, 2024; v1 submitted 26 October, 2023; originally announced October 2023.

Comments: Accepted at SaTML 2024

Exploring the Potential of Generative AI for the World Wide Web

Authors: Nouar AlDahoul, Joseph Hong, Matteo Varvello, Yasir Zaki

Abstract: Generative Artificial Intelligence (AI) is a cutting-edge technology capable of producing text, images, and various media content leveraging generative models and user prompts. Between 2022 and 2023, generative AI surged in popularity with a plethora of applications spanning from AI-powered movies to chatbots. In this paper, we delve into the potential of generative AI within the realm of the Worl… ▽ More Generative Artificial Intelligence (AI) is a cutting-edge technology capable of producing text, images, and various media content leveraging generative models and user prompts. Between 2022 and 2023, generative AI surged in popularity with a plethora of applications spanning from AI-powered movies to chatbots. In this paper, we delve into the potential of generative AI within the realm of the World Wide Web, specifically focusing on image generation. Web developers already harness generative AI to help crafting text and images, while Web browsers might use it in the future to locally generate images for tasks like repairing broken webpages, conserving bandwidth, and enhancing privacy. To explore this research area, we have developed WebDiffusion, a tool that allows to simulate a Web powered by stable diffusion, a popular text-to-image model, from both a client and server perspective. WebDiffusion further supports crowdsourcing of user opinions, which we use to evaluate the quality and accuracy of 409 AI-generated images sourced from 60 webpages. Our findings suggest that generative AI is already capable of producing pertinent and high-quality Web images, even without requiring Web designers to manually input prompts, just by leveraging contextual information available within the webpages. However, we acknowledge that direct in-browser image generation remains a challenge, as only highly powerful GPUs, such as the A40 and A100, can (partially) compete with classic image downloads. Nevertheless, this approach could be valuable for a subset of the images, for example when fixing broken webpages or handling highly private content. △ Less

Submitted 26 October, 2023; originally announced October 2023.

Comments: 11 pages, 9 figures

Ampleness of normal bundles of base cycles in flag domains

Authors: Jaehyun Hong, Aeryeong Seo

Abstract: Flag domains are open orbits of noncompact real forms of complex semisimple Lie groups acting on flag manifolds. To each flag domain one can associate a compact complex manifold called the base cycle. The ampleness of the normal bundle of the base cycle in a flag domain measures the concavity near the base cycle. In this paper we compute the ampleness of normal bundles of base cycles in flag domai… ▽ More Flag domains are open orbits of noncompact real forms of complex semisimple Lie groups acting on flag manifolds. To each flag domain one can associate a compact complex manifold called the base cycle. The ampleness of the normal bundle of the base cycle in a flag domain measures the concavity near the base cycle. In this paper we compute the ampleness of normal bundles of base cycles in flag domains in various cases, including flag domains in the full flag manifolds $G/B$ when $G$ is classical, and period domains parameterizing polarized Hodge structures with fixed Hodge numbers. △ Less

Submitted 25 October, 2023; originally announced October 2023.

Comments: 27 pages

MSC Class: 14M15; 32M05; 57S20

CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignment

Authors: Jixiang Hong, Quan Tu, Changyu Chen, Xing Gao, Ji Zhang, Rui Yan

Abstract: Language models trained on large-scale corpus often generate content that is harmful, toxic, or contrary to human preferences, making their alignment with human values a critical concern. Reinforcement learning from human feedback (RLHF) with algorithms like PPO is a prevalent approach for alignment but is often complex, unstable, and resource-intensive. Recently, ranking-based alignment methods h… ▽ More Language models trained on large-scale corpus often generate content that is harmful, toxic, or contrary to human preferences, making their alignment with human values a critical concern. Reinforcement learning from human feedback (RLHF) with algorithms like PPO is a prevalent approach for alignment but is often complex, unstable, and resource-intensive. Recently, ranking-based alignment methods have emerged, offering stability and effectiveness by replacing the RL framework with supervised fine-tuning, but they are costly due to the need for annotated data. Considering that existing large language models (LLMs) like ChatGPT are already relatively well-aligned and cost-friendly, researchers have begun to align the language model with human preference from AI feedback. The common practices, which unidirectionally distill the instruction-following responses from LLMs, are constrained by their bottleneck. Thus we introduce CycleAlign to distill alignment capabilities from parameter-invisible LLMs (black-box) to a parameter-visible model (white-box) in an iterative manner. With in-context learning (ICL) as the core of the cycle, the black-box models are able to rank the model-generated responses guided by human-craft instruction and demonstrations about their preferences. During iterative interaction, the white-box models also have a judgment about responses generated by them. Consequently, the agreement ranking could be viewed as a pseudo label to dynamically update the in-context demonstrations and improve the preference ranking ability of black-box models. Through multiple interactions, the CycleAlign framework could align the white-box model with the black-box model effectively in a low-resource way. Empirical results illustrate that the model fine-tuned by CycleAlign remarkably exceeds existing methods, and achieves the state-of-the-art performance in alignment with human value. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Intuitive Multilingual Audio-Visual Speech Recognition with a Single-Trained Model

Authors: Joanna Hong, Se Jin Park, Yong Man Ro

Abstract: We present a novel approach to multilingual audio-visual speech recognition tasks by introducing a single model on a multilingual dataset. Motivated by a human cognitive system where humans can intuitively distinguish different languages without any conscious effort or guidance, we propose a model that can capture which language is given as an input speech by distinguishing the inherent similariti… ▽ More We present a novel approach to multilingual audio-visual speech recognition tasks by introducing a single model on a multilingual dataset. Motivated by a human cognitive system where humans can intuitively distinguish different languages without any conscious effort or guidance, we propose a model that can capture which language is given as an input speech by distinguishing the inherent similarities and differences between languages. To do so, we design a prompt fine-tuning technique into the largely pre-trained audio-visual representation model so that the network can recognize the language class as well as the speech with the corresponding language. Our work contributes to developing robust and efficient multilingual audio-visual speech recognition systems, reducing the need for language-specific models. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Comments: EMNLP 2023 Findings

Can CNNs Accurately Classify Human Emotions? A Deep-Learning Facial Expression Recognition Study

Authors: Ashley Jisue Hong, David DiStefano, Sejal Dua

Abstract: Emotional Artificial Intelligences are currently one of the most anticipated developments of AI. If successful, these AIs will be classified as one of the most complex, intelligent nonhuman entities as they will possess sentience, the primary factor that distinguishes living humans and mechanical machines. For AIs to be classified as "emotional," they should be able to empathize with others and cl… ▽ More Emotional Artificial Intelligences are currently one of the most anticipated developments of AI. If successful, these AIs will be classified as one of the most complex, intelligent nonhuman entities as they will possess sentience, the primary factor that distinguishes living humans and mechanical machines. For AIs to be classified as "emotional," they should be able to empathize with others and classify their emotions because without such abilities they cannot normally interact with humans. This study investigates the CNN model's ability to recognize and classify human facial expressions (positive, neutral, negative). The CNN model made for this study is programmed in Python and trained with preprocessed data from the Chicago Face Database. The model is intentionally designed with less complexity to further investigate its ability. We hypothesized that the model will perform better than chance (33.3%) in classifying each emotion class of input data. The model accuracy was tested with novel images. Accuracy was summarized in a percentage report, comparative plot, and confusion matrix. Results of this study supported the hypothesis as the model had 75% accuracy over 10,000 images (data), highlighting the possibility of AIs that accurately analyze human emotions and the prospect of viable Emotional AIs. △ Less

Submitted 13 October, 2023; originally announced October 2023.

ACM Class: I.2.6; I.4.m