Tutorials

Selected Tutorials

  1. Compressing Large Language Models
  2. Mining Multi-layer Graphs: Models, Algorithms and Applications
  3. Redefining Generative Modeling in Data-limited Environments
  4. Data Fabric: Technologies and Applications
  5. Secure and Private Federated Learning with Differential Privacy
  6. Real-time Forecasting of Big Time-series Data: Foundations and Challenges

Tutorial 1

Compressing Large Language Models

Speaker

U Kang (Seoul National University)

Place

2F Conference Room 5

Date and Time

2:00 p.m. – 3:30 p.m. on Wednesday, July 3rd

Brief Outline of the Tutorial

How can we compress large language models efficiently? Deep learning is one of the most widely used machine learning techniques, and is a key driving force of the 4th industrial revolution. Deep learning outperforms many existing algorithms and even humans especially for many difficult tasks including speech recognition, go, language translation, game, etc. Recently, large language models have appeared as one of the key areas of deep learning with many applications including machine translation, sentiment analysis, question answering, etc. One crucial challenge of large language models is its efficiency both in training and inference. Large language models require a lot of parameters which need huge amount of time and space. Such inefficiency makes many problems, including huge energy requirement, huge running time, huge memory and disk apace, and difficulty for deployment in mobile devices. Our goal in this tutorial is to describe model compression techniques for large language models to solve such problems.

We start with a very brief background of deep learning and large language models, including its history, application, and popular models including Transformer and BERT. Then we describe how to compress large language models using techniques including pruning, quantization, knowledge distillation, and parameter-efficient fine tuning. We expect to give audience substantial knowledge about reducing time and space in using large language models.

Bio

U Kang is a Professor in the Department of Computer Science and Engineering of Seoul National University. He received Ph.D. in Computer Science at Carnegie Mellon University, after receiving B.S. in Computer Science and Engineering at Seoul National University. He won 2013 SIGKDD Doctoral Dissertation Award, 2013 New Faculty Award from Microsoft Research Asia, 2016 Korean Young Information Scientist Award, and six “best paper” awards including 2018 ICDM 10-year best paper award, 2021 KDD best research paper award, and 2022 ICDE best research paper award. He has published over 100 refereed articles in major data mining, database, and machine learning venues. He holds seven U.S. patents. His research interests include data mining and machine learning.

Tutorial 2

Mining Multi-layer Graphs: Models, Algorithms and Applications

Speakers

  • Zhaonian Zou (Harbin Institute of Technology)
  • Dandan Liu (Harbin Institute of Technology)
  • Run-An Wang (Harbin Institute of Technology)

Place

2F Conference Room 5

Date and Time

2:00 p.m. – 3:30 p.m. on Thursday, July 4th

Brief Outline of the Tutorial

Multi-layer graph has emerged as a new representation of complex relationships between entities from various domains in the real world. Mining dense substructures in multi-layer graphs can provide deeper insights into the underlying complex systems. In this tutorial, we will introduce the models of multi-layer graphs, present various dense substructures in multi-layer graphs, describe the algorithms for mining dense substructures from multi-layer graphs, and demonstrate the applications of dense substructures in various applications.

Bio

Zhaonian Zou is a professor in the School of Computer Science and Technology, Harbin Institute of Technology, China and an associate director at the Heilongjiang Provincial Lab on Big Data Science and Engineering. He completed his B.Sc. and M.Eng. degrees in computer science from Jilin University, China in 2002 and 2005, respectively, and obtained his Ph.D. degree in computer science from Harbin Institute of Technology in 2010. He is the principal investigator of 3 projects granted by the National Natural Science Foundation of China. Dr. Zou’s research interests lie in the area of big data and database systems. He has published more than 60 papers in VLDB, ICDE, KDD, TKDE, VLDBJ, etc.

Dandan Liu is a Ph.D. candidate in computer science at the School of Computer Science and Technology, Harbin Institute of Technology, China under the supervision of Prof. Zhaonian Zou. She received her bachelor degree in computer science from Harbin Institute of Technology in 2019. Her primary research interests lie in the area of graph mining and graph databases. She has published 4 papers in VLDB, ICDE and WWW Journal.

Run-An Wang is a second year Ph.D. student in computer science at the School of Computer Science and Technology, Harbin Institute of Technology, China under the direction of Prof. Zhaonian Zou. He completed his bachelor degree in computer science from Beihang University, China in 2020 and obtained his M.Eng. degree in computer science from Harbin Institute of Technology in 2022. His primary research interests include graph mining and database systems. He has published 3 papers in ICDE and Information Systems.

Tutorial 3

Redefining Generative Modeling in Data-limited Environments

Speakers

  • Divya Saxena (The Hong Kong Polytechnic University)
  • Jiannong Cao (The Hong Kong Polytechnic University)
  • Tarun Kulshrestha (The Hong Kong Polytechnic University)

Place

2F Conference Room 5

Date and Time

4:00 p.m. – 5:30 p.m. on Thursday, July 4th

Brief Outline of the Tutorial

Generative models have showcased remarkable applicability, particularly synthesizing realistic data, uncovering patterns, and enhancing decision-making across a variety of domains. Despite their proven success, the application of these models faces notable challenges in environments where data collection is constrained by privacy, cost, or ethical concerns. This tutorial seeks to address these challenges by providing strategies for adapting to scenarios where data is limited. Designed to empower participants, it offers the tools needed to navigate and innovate within these constraints. In this tutorial, we will explore three key areas: (1) We will cover foundational concepts of models such as GANs, VAEs, and diffusion models, emphasizing the challenges of limited data availability and the trade-offs between model complexity and data constraints; (2) We will delve into recent innovations including transfer learning and data augmentation, discussing how these can improve performance in data-scarce environments and highlighting efficient architectural designs; (3) The tutorial will conclude with case studies from various industries, demonstrating the practical application of these models and providing insights into how these techniques can be implemented effectively in different sectors.

Bio

Divya Saxena is working as Research Assistant Professor in the Department of Computing at The Hong Kong Polytechnic University, Hong Kong. Her current research interests include deep generative modelling, efficient and scalable vision, spatio-temporal prediction and continual learning. She has published more than 10 refereed papers in various journals and conferences.

Jiannong Cao is currently the Otto Poon Charitable Foundation Professor in Data Science and the Chair Professor of Distributed and Mobile Computing in the Department of Computing at The Hong Kong Polytechnic University, Hong Kong. He is also the Dean of Graduate School, the Director of Research Institute for Artificial Intelligence of Things, and Director of Internet and Mobile Computing Lab. His research interests include Distributed Systems and Blockchain, Big data and Machine learning, Wireless Sensing and Networking, and Mobile Cloud and Edge Computing. He has published more than 100+ refereed papers in various journals and conferences.

Tarun Kulshrestha is currently working as a Research Fellow in the UBDA at The Hong Kong Polytechnic University, Hong Kong. His current research interests include deep generative modelling and efficient and scalable vision. He has published more than 10 refereed papers in various journals and conferences.

Tutorial 4

Data Fabric: Technologies and Applications

Speakers

  • Kamalakar Karlapalem (IIIT Hyderabad)
  • P Radha Krishna (National Institute of Technology, Warangal)
  • Satyanarayana R Valluri (Databricks Inc.)

Place

2F Conference Room 5

Date and Time

2:00 p.m. – 3:30 p.m. on Friday, July 5th

Brief Outline of the Tutorial

Data platforms and storage technologies, including data mesh, data lakes, data warehouses, and cloud and federated databases, manage vast amounts of data from diverse distributed sources for queries and analytics. Data fabric, a recent development in data management, offers a comprehensive architecture to seamlessly integrate disparate data pipelines, reducing latency and ensuring robust data governance. Leveraging distributed computing capabilities, the data fabric architecture manages data, query, and analytics pipelines without moving data to a centralized location. Despite its potential, many application designers lack the necessary perspectives and knowledge of metadata, domain dependencies, and performance parameters. Additionally, understanding the inter- and intra-relationships among metadata associated with source systems, the data fabric, and the application domain is essential. Addressing this knowledge gap is crucial for the successful implementation and utilization of data fabric solutions. In this tutorial, we address this gap by providing a comprehensive lesson on the background technologies of data fabric and the need for metadata to comprehend and develop applications on top of data fabric. We shall give examples using Microsoft Fabric and present how metadata plays a directed role in developing these applications. Finally, we present the need for better comprehension of metadata with suitable case scenarios.

Bio

Kamal Karlapalem is a Professor and Head of the Data Science and Analytics Center IIIT-Hyderabad. He had worked for over three decades on specific problems of distributed relational database design, object-oriented database partitioning and allocation, and larger data warehouse design. He introduced the problem of a total redesign of distributed relational databases in 1992 and worked on distributed data systems and their design for the last few decades. Currently, he has been developing conceptual modeling frameworks to support the data fabric solutions. His research interests include database systems, data visualization, data analytics, multi-agent systems, workflows, and electronic contracts.

Radha Krishna is a Professor at the Department of Computer Science and Engineering, National Institute of Technology Warangal. Prior to joining NIT, he worked at Infosys Labs, IDRBT, and National Informatics Centre (Govt. of India), where he was associated with research projects leading to futuristic intelligent systems and analytical solutions. He holds PhDs from Osmania University and IIIT-Hyderabad. His research interests include data mining, big data, machine learning, databases, and e-contracts & services.

Satya Valluri is a Software Engineer at Databricks, USA. He is part of the query optimizer group and focuses on optimizing SQL queries for Spark and Databricks SQL. Previously, he worked at Meta Platforms Inc, USA, where he was involved in managing a highly scalable and distributed database that stores the operational data of Meta. Before Meta, Satya worked in the Query Optimizer group of Oracle. His main areas of interest are query processing and optimization, query execution and manageability, and debuggability of features in DBMS systems. Satya did a postdoctoral fellowship at EPFL, Switzerland, and has a Ph.D. from IIIT, Hyderabad.

Tutorial 5

Secure and Private Federated Learning with Differential Privacy

Speakers

Tsubasa Takahashi (LY Corp.)

Place

2F Conference Room 5

Date and Time

11:00 a.m. – 12:30 p.m. on Friday, July 5th

Brief Outline of the Tutorial

In an era where artificial intelligence (AI) is increasingly integrated into various aspects of life, the imperative of ensuring data privacy cannot be overstated. Federated learning (FL) is a scalable and privacy-centric framework designed for large-scale data analysis and machine learning. To provide rigorous privacy guarantees, recent practices in FL have incorporated differential privacy (DP). The implementations of DP-enhanced FL vary based on different trust assumptions. This session will explore the foundational principles and practical applications of differentially private federated learning, highlighting recent advancements that incorporate secure aggregation techniques.

Bio

Tsubasa Takahashi is a research scientist affiliated with LY Corporation and SB Intuitions Corp. He got his PhD in Computer Science from the University of Tsukuba in 2014. His career includes a tenure as a research scientist at NEC Corporation from 2010 to 2018, and he spent a year as a visiting scholar at Carnegie Mellon University from 2015 to 2016. His research interests are centered around data mining, data privacy, and the development of trustworthy machine learning systems. He has contributed to over 50 publications in various international conferences and journals, such as SIGMOD, VLDB, WWW, CVPR, and ICLR.

Tutorial 6

Real-time Forecasting of Big Time-series Data: Foundations and Challenges

Speakers

  • Yasushi Sakurai (Osaka University)
  • Yasuko Matsubara (Osaka University)

Place

2F Conference Room 5

Date and Time

4:00 p.m. – 5:30 p.m. on Wednesday , July 3rd

Brief Outline of the Tutorial

The emergence of various types of sensors, particularly Internet of Things (IoT) devices, has garnered considerable attention across diverse fields such as manufacturing, mobility, medical science, healthcare, environmental protection, and more. Sensors are small devices that gather a huge amount of data. Faced with such circumstances, one of the biggest challenges is to develop real-time data mining technologies for time-series data, specifically smart analytical tools that capture the latent time evolution. This tutorial provides a concise and intuitive overview of the most important tools that we can use to help us understand and find patterns in large-scale time evolving sequences. We also provide the intuition behind fundamental technologies for the real-time modeling and forecasting of big data streams, as well as to introduce case studies that illustrate their practical use.

Bio

Yasushi Sakurai is a professor at Osaka University, Japan. He received a B.E. degree from Doshisha University in 1991, and M.E. and Ph.D. degrees from Nara Institute of Science and Technology in 1996 and 1999, respectively. He joined NTT Laboratories in 1998. He was a visiting researcher at Carnegie Mellon University during 2004-2005. During 2013-2019, he was a Professor at Kumamoto University. He received two KDD best paper awards in 2008 and 2010. His research interests include time-series analysis, web mining, and sensor data processing.

Yasuko Matsubara is an Associate Professor at Osaka University, Japan. She received her BS and MS degrees from Ochanomizu University in 2007 and 2009 respectively, and her Ph.D. from Kyoto University in 2012. She was a visiting researcher at Carnegie Mellon University in 2011-2012 and 2013-2014. She was an Assistant Professor at Kumamoto University, during 2014-2019. She has received the IPSJ/ACM Award for Early Career Contributions to Global Research (2018), ACM Recognition of Service Award (2020). Her research interests include time-series data mining and nonlinear dynamic systems.