About

Girolamo Macaluso
PhD Student in Artificial Intelligence
Media Integration and Communication Center, University of Florence, Italy

📍 Florence, Italy · ✉️ girolamo.macaluso@unifi.it · 🔗 LinkedIn · 💻 GitHub · 🎓 Scholar

My research is dedicated to making Reinforcement Learning more efficient and scalable. I began by exploring offline and hybrid offline→online RL methods to maximize learning from static datasets, then developed techniques to cut the computational and memory costs of state-of-the-art algorithms. I’m now advancing continual RL frameworks that adapt to evolving environments and applying RL to fine-tune diffusion models.

---

Education

• PhD in Artificial Intelligence
  Nov 2023–Present · University of Florence
  Topic: Efficient Reinforcement Learning for game development.

• MSc in Artificial Intelligence
  Sep 2021–Jul 2023 · University of Florence
  Thesis: Enhancing Offline RL with Generative World Models
  Grade: 110 L/110 (GPA 4.0)

• BSc in Computer Engineering
  Sep 2018–Jul 2021 · University of Florence

---

Experience

Part-time Software Engineer, Maba S.R.L, Florence
Jan 2019–Jul 2023
Designed and delivered end-to-end enhancements to a centralized ERP platform, including database modeling, backend API development, and user-facing interface components to support customer-specific workflows.

---

Research Interests

• Offline Reinforcement Learning: leverage pre-collected data without online interaction
• Fine-tuning of Diffusion Models: adapt diffusion models to different tasks with reinforcement learning
• Continual Reinforcement Learning: create agents that are capable to adapt to changes of the environment
• Reinforcement Learning for Game Development: accelerate and improve video game development with reinforcement learning

---

Publications

1. SPEQ: Offline Stabilization Phases for Efficient Q-Learning in High Update-To-Data Ratio RL.
   Accepted at Reinforcement Learning Conference 2025. 🔗Open Review 🔗GitHub (Code available soon)
   Introduces a novel online reinforcement learning framework with periodic offline stabilization that cuts computational requirements and training time while matching or exceeding state-of-the-art performance.

2. A Benchmark Environment for Offline RL in Racing Games.
   Oral at IEEE Conference on Games 2024. 🔗Arxiv 🔗GitHub
   Introduces OfflineMania a novel benchmark environment for offline and offline to online RL with challenging data distribution.

3. Small Dataset, Big Gains: Enhancing RL by Offline Pre-Training with Model-Based Augmentation.
   Oral at AIBSD Workshop, AAAI 2024. 🔗Arxiv
   Proposed a novel technique based on a generative model on for improving offline to online RL performance when dealing with small datasets.

---

Teaching & Mentoring

Teaching Assistant, University of Florence
Jan 2023–Present
Interactive C/C++ & Python lessons for 200+ undergraduates.

STEM Outreach Teacher, European STEM Project
Feb 2024–Present
Introduce students to the field of engineering and explain the challenges of applying AI to real world problems.