Atsuyuki Miyai 宮井 淳行

I'm Atsuyuki Miyai. Now, I'm a 1st-year PhD student of the Yamasaki Lab at The University of Tokyo 🇯🇵.

Before that, I received my master degree under the supervision of Prof. Kiyoharu Aizawa at UTokyo. With his retirement approaching, I have moved to the Yamasaki Lab, but I will continue collaborative research with the Aizawa Lab and tackle with the same research topic.

My research focuses on the safety of foundation models. In particular, I have contributed to the development of the field of Out-of-Distribution Detection for Vision Language Models (LoCoOp) and the evolution to Large Multi-modal Models (UPD).
I also aim to contribute to recent efforts for the hallucination issues and the development of multilingual LMMs (in a Japanese part), to help realize a future where many users can trust and comfortably use AI systems.

I want to put an emphasis on defining new problems and aim to provide a simple and baseline approach to give insights.

I am seeking internships in industry or academia in 2025, focusing on the trustworthy LMMs/LLMs. Do not hesitate to contact me if there's a good fit 🤝.

The University of Tokyo
Ph.D. in Graduate School of Information Science and Technology
Supervisor: Prof. Toshihiko Yamasaki
2024.4 -

The University of Tokyo
M.S. in Graduate School of Interdisciplinary Information Studies
Supervisor: Prof. Kiyoharu Aizawa
2022.4 - 2024.3

The University of Tokyo
B.S. in Under Graduate School of Engineering
Information and Communication Engineering
Supervisor: Prof. Kiyoharu Aizawa
2018.4 - 2022.3

2024.10: 🎉 We introduce JMMMU (Japanese MMMU)! Lets's try your favorite LMM with our JMMMU!

2024.8: 🎉 Our new survey "Generalized OOD detection v2", has been out!

2024.7: I will give an invited talk on LoCoOp at MIRU2024 (Most prestigious Japanese CV conf.). 是非、オーラル/ポスター会場にお越しください!

2024.4: 🎉 Our new preprint, Unsolvable Problem Detection, has been out in arXiv!

2024.4: Doctoral program begins at the University of Tokyo.

International

2024

• JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark
  Shota Onohara*, Atsuyuki Miyai*, Yuki Imajuku*, Kazuki Egashira*, Jeonghun Baek*, Xiang Yue,
  Graham Neubig, Kiyoharu Aizawa
  arXiv, 2024
  NeurIPS EvalEval Workshop Oral Presentation, 2024 (preliminary version)
  paper /  project page /  dataset /  code /  leaderboard
  
  
  
  
• Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey
  Atsuyuki Miyai, Jingkang Yang, Jingyang Zhang, Yifei Ming, Yueqian Lin,
  Qing Yu, Go Irie, Shafiq Joty, Yixuan Li, Hai Li, Ziwei Liu, Toshihiko Yamasaki,
  and Kiyoharu Aizawa
  arXiv, 2024
  paper  /  code
  
  
  
  
• Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models
  Atsuyuki Miyai, Jingkang Yang, Jingyang Zhang, Yifei Ming,
  Qing Yu, Go Irie, Yixuan Li, Hai Li, Ziwei Liu, and Kiyoharu Aizawa
  arXiv, 2024
  ICLR Workshop on Reliable and Responsible Foundation Models, 2024 (preliminary version)
  paper  /  code  /  dataset  /  leaderboard

• Can Pre-trained Networks Detect Familiar Out-of-Distribution Data?
  Atsuyuki Miyai, Qing Yu, Go Irie, and Kiyoharu Aizawa
  arXiv, 2023
  paper  /  code (comming soon)




• LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning
  Atsuyuki Miyai, Qing Yu, Go Irie, and Kiyoharu Aizawa
  Neural Information Processing Systems (NeurIPS), 2023
  paper  /  code




• Zero-Shot In-Distribution Detection in Multi-Object Settings Using Vision-Language Foundation Models
  Atsuyuki Miyai, Qing Yu, Go Irie, and Kiyoharu Aizawa
  arXiv, 2023
  paper  /  code



• Rethinking Rotation in Self-Supervised Contrastive Learning:
  Adaptive Positive or Negative Data Augmentation
  Atsuyuki Miyai, Qing Yu, Daiki Ikami, Go Irie, and Kiyoharu Aizawa
  Winter Conference on Applications of Computer Vision (WACV), 2023
  paper  /  code

• Unsolvable Problem Detection: Towards Trustworthy Vision-Language Models
  Atsuyuki Miyai, Jingkang Yang, Jingyang Zhang, Yifei Ming,
  Qing Yu, Go Irie, Yixuan Li, Hai Li, Ziwei Liu, and Kiyoharu Aizawa
  MIRU, 2024
  
  
• CLIPを用いたゼロショット分布内検知
  宮井 淳行, 郁 青, 入江 豪, 相澤 清晴
  IE, 2024, IE Award
  
  
• CLIP-based In-distribution Detection
  Atsuyuki Miyai, Qing Yu, Go Irie, Kiyoharu Aizawa
  MIRU, 2023 (short oral)
  
  
• Rethinking Rotation for Contrastive Learning: Adaptive Positive or Negative Data Augmentation
  Atsuyuki Miyai, Qing Yu, Daiki Ikami, Go Irie, Kiyoharu Aizawa
  MIRU, 2022 (long oral), Best Student Paper
  
  
• 自己教師あり学習のための適応的正負例データ拡張
  宮井 淳行, 郁 青, 伊神大貴, 入江 豪, 相澤 清晴
  IPSJ (情報処理学会), 2022, Student Encouragement Award
  

2024.8: Invited Talk for LoCoOp @MIRU 2024

2024.3: The Dean's Award, The Best Master Thesis Award

2024.2: IE Award @IE (画像工学研究会) Feb. 2024

2023.7: Interactive Presentation Award @MIRU 2023

2022.7: Best Student Paper Award @MIRU 2022
Awarded The Best Student Paper Award by The Association for Meeting on Image Recognition and Understanding (Japanese representative scientific and professional society for researchers working on problems involving computer vision).
Please refer to the official homepage.

2022.3: Student Encouragement Award @Information Processing Society of Japan (IPSJ) 2022

2023.6 ~ 2023.12: OMRON SINICX
Research Internship.   Mentor: Dr. Yoshitaka Ushiku, Dr. Atsushi Hashimoto, Dr. Kuniaki Saito

2022.8 ~ 2022.9: Preferred Networks
Research Internship.   Mentor: Dr. Masanori Koyama, Dr. Kohei Hayashi, Dr. Yuta Kikuchi, Dr. Yoshihiro Yamada

2020.10 ~ 2023.03: Pluszero Inc.
Project Engineer.  I tackle natural language processing.

2023.12 ~ : OpenOOD
Research activities.   Collaborators: Jingkang Yang (Nanyang Technological University), Jingyang Zhang (Duke University), Yifei Ming (Salesforce Research)

2023.4 ~ : IEEE Student Branch Chair @UTokyo
We have a diverse range of workshops, invited talks, and hands-on activities focused on advancing knowledge and skills in a particular field or topic. Please feel free to contact us when you visit Tokyo!
Please refer to the official homepage.