Course Syllabus

Overview

This advanced course delves into the exciting field of Multi-modalities Generative AI, where students will explore the convergence of multiple data modalities such as text, image, and audio to create sophisticated and expressive AI models. This course is divided into two equally important components. The first half will consist of a lecture series that systematically covers fundamental concepts and technologies in generative AI. This segment aims to establish a solid theoretical foundation while facilitating hands-on learning through practical demonstrations with open-source examples. The second half of the course will be an interactive reading group, where students delve into influential papers and engage in in-depth discussions. This segment is designed to foster critical thinking, encourage exploration of the latest research in multi-modal generative AI.

Students will undertake a term project directly related to Multi-modalities Generative AI. This project will provide an opportunity to apply the acquired knowledge and skills, fostering creativity and innovation in the development of multi-modal generative models. By combining a lecture series, engaging reading group discussions, and a practical term project, this course aims to equip students with a well-rounded understanding of multi-modal generative AI and the ability to contribute meaningfully to this rapidly evolving field.

Prerequisites

• Background in Deep Learning and Machine Learning
• or Instructor's Permission

Students will need to be comfortable with concepts from deep learning and machine learning. Excellent programming skills are necessary!

Class Meetings:

Location: Engineering & CS Center (ECSC) Room 116 ; 

Lecture: Tuesday/Thursday, 4:30 – 6:20 pm

X-hour: Friday, 4:35 – 5:25 pm

 

Format/Attendance

This course will be in-person with some online components.

I expect you to attend class in person unless you have made alternative arrangements due to illness, medical reasons, or the need to isolate due to COVID-19. For the health and safety of our class community, please: do not attend class when you are sick, nor when you have been instructed by Student Health Services to stay home.

Course staff

Instructor: Yu-Wing Tai <yu-wing.tai@dartmouth.edu>

Teaching Assistant: Wayner Barrios Quiroga <wayner.j.barrios.quiroga.gr@dartmouth.edu>

Office Hours

You can make an appointment with the course staff (but keep in mind it may take some time to schedule). 

• Yu-Wing Tai: Tuesday / Thursday 3:00-4:30pm ECSC 208 or Appointment via Email
• Wayner Barrios Quiroga: Thursday 1:00-3:00pm/Friday 1:00-2:00pm ECSC 206, Zoom (https://dartmouth.zoom.us/my/waybarrios)  or Appointment via Email

If you find you need additional, individualized help beyond office hours, you can reach out to Dartmouth's Tutor Clearinghouse.

Lectures & Coursework

This is a seminar course, and the majority of your grade will come from class presentation and term project. 

Graduate

The undergraduate (89) and graduate (189) sections of this class will be graded separately and we will generally grade graduate students more strictly. For graduates enrolled in 189, I will convert an A to High Pass (HP) and B- or above to Pass (P). Low Pass (LP) starts at C+, and anything D or below receives No Credit (NC).

Submission Deadlines

You will turn in the submission through Canvas, and each assignment will have a strict deadline. 

It’s up to you to check that assignments have been successfully submitted to Canvas. Don’t upload seconds before the deadline to avoid accidents. Double-check that you indeed uploaded the correct file.

Late policy

Late is as defined by Canvas.

For other coursework, there is a 4% per (any portion of an) hour deduction for late submissions. This means that if you submit 70 minutes late, the maximum score you can receive is 92%, and submissions more than 24 hours late get a zero.

Late days:

You also have 2 free "late days" to use during the term. A late day can be used to waive the late penalty for any portion of a day on reading assignments, but not the presentation and term project. Late days cannot be used after the last day of class.

To use a late day and avoid incurring the late deduction, you must post a comment in canvas along with your submission stating: "I would like to use a late day for this."

Exceptions & extensions:

I understand that life and health sometimes gets in the way of our academic responsibilities. Generally speaking I am quite understanding if you reach out to me in advance, but not if you reach out to me after the fact. If you believe you may need an extension, then reach out to me in writing well before the deadline (e.g. >1 week, not a couple days). If you reach out to me after the deadline, I will only grant extensions in the case of a medical emergency.

Regrades

Grading is a noisy process, so there may sometimes be errors (in either direction).

If you believe there is an error in the grading, first make sure you understand everything in the assignment. You may then submit a regrade request by emailing the course staff explaining your concern. We will then regrade your entire assignment or quiz, not just the portion you believe is in error. If we find previously missing deductions, your score may actually go down.

I will only consider regrades that change your grade by at least a full letter grade.

We will not consider regrade requests submitted more than 1 week after the grade is posted, or after the last day of class. 

Honor Principle and permissible sources of information

If you are not already, you should familiarize yourself with Dartmouth's policies on the Academic Honor Principle, and Proper Citation of Sources.

In short: You are welcome and encouraged to chat about assignments with other students in general terms, but your solutions must be written and developed on your own.

Using tools like Copilot and ChatGPT to generate (portions) of your solution is a violation of the honor principle.

Properly attributing outside sources (if any)

If you’re ever in doubt, just include a citation in your code and report indicating where some idea came from, whether it be a classmate, a website, another piece of software, or anything. Note that you may still receive a zero for that task if you simply copy the solution from some outside source, but citing it at least maintains your honesty and would not be considered a violation of the academic honor principle.

You should treat proper attribution in code just like you would if you were writing an essay or journal article.

In this class, proper citation format in your code should include two things:

1. At the top of the file include a comment block listing all instances of outside sources used within that file. Think of this like the Works Cite/Bibliography section of a journal article. Number the references so you can refer to them later in the source code. For each item, list the URL, copyright notice any license if applicable, as well as information about how the code/idea is used.
2. Include a comment at a point close to where you adopt some idea/code from somewhere else. Think of this like an in-text parenthetical citation in a journal article. This should include enough information so that it is clear which outside source from the top of your file is being referenced (e.g. the reference number, or the name of author or library). 

The same basic principle applies to your presentations. Any material you reuse from outside sources must be properly attributed both on the slide it is used, and in a Bibliography/References slide.

 

Attendance

You are expected to attend class in person unless you have made alternative arrangements due to illness or other medical reasons. For the health and safety of our class community, please: do not attend class when you are sick, nor when you have been instructed by Student Health Services to stay home.

Consent to recording

Dartmouth has asked that I include the following language describing some bounds on how recording should be used. You do not need to send me any sort of agreement on this – it just makes clear that you shouldn’t record or distribute any recordings without my consent.

The remainder is standard text provided by Dartmouth [here].

1. Consent to recording of course meetings and office hours that are open to multiple students. By enrolling in this course,

   1. I affirm my understanding that the instructor may record meetings of this course and any associated meetings open to multiple students and the instructor, including but not limited to scheduled and ad hoc office hours and other consultations, within any digital platform, including those used to offer remote instruction for this course.

   2. I further affirm that the instructor owns the copyright to their instructional materials, of which these recordings constitute a part, and my distribution of any of these recordings in whole or in part to any person or entity other than other members of the class without prior written consent of the instructor may be subject to discipline by Dartmouth up to and including separation from Dartmouth.

2. Requirement of consent to one-on-one recordings

   By enrolling in this course, I hereby affirm that I will not make a recording in any medium of any one-on-one meeting with the instructor or another member of the class or group of members of the class without obtaining the prior written consent of all those participating, and I understand that if I violate this prohibition, I will be subject to discipline by Dartmouth up to and including separation from Dartmouth, as well as any other civil or criminal penalties under applicable law. I understand that an exception to this consent applies to accommodations approved by SAS for a student's disability, and that one or more students in a class may record class lectures, discussions, lab sessions, and review sessions and take pictures of essential information, and/or be provided class notes for personal study use only.

If you have questions, please contact the Office of the Dean of the Faculty of Arts and Sciences.

Accommodations

Students with disabilities who may need disability-related academic adjustments and services for this course should see me privately within the first week of class.

The remainder is standard text provided by Dartmouth [here].

Students requesting disability-related accommodations and services for this course are required to register with Student Accessibility Services (SAS; Getting Started with SAS webpage; student.accessibility.services@dartmouth.edu; 1-603-646-9900) and to request that an accommodation email be sent to me in advance of the need for an accommodation. Then, students should schedule a follow-up meeting with me to determine relevant details such as what role SAS or its Testing Center may play in accommodation implementation. This process works best for everyone when completed as early in the quarter as possible. If students have questions about whether they are eligible for accommodations or have concerns about the implementation of their accommodations, they should contact the SAS office. All inquiries and discussions will remain confidential.

Mental Health and Awareness

The following is standard text provided by Dartmouth [here].

The academic environment at Dartmouth is challenging, our terms are intensive, and classes are not the only demanding part of your life. There are a number of resources available to you on campus to support your wellness, including your undergraduate dean, Counseling and Human Development, and the Student Wellness Center. I encourage you to use these resources to take care of yourself throughout the term, and to come speak to me if you experience any difficulties.

Respect, Diversity, and Inclusion

I would like to create a learning environment for my students that supports a diversity of thoughts, perspectives and experiences, and honors your identities (including race, gender, class, sexuality, religion, ability, etc.). To help accomplish this:

If you have a name and/or set of pronouns that differ from those that appear in your official college records, please include them in DartHub or let me know privately.

If at any time you feel uncomfortable about the interactions in our classroom I encourage you to contact me privately so I can better understand how I can manage the course; indeed, I am eager for feedback about how I can maximize everyone’s experience. If you feel like your performance in the class is being impacted by your experiences outside of class, likewise, please don’t hesitate to contact me. I want to be a resource for you. If you prefer to speak with someone outside of the course, the contacts in the Mental Health and Wellness section above can be an excellent resource.

I (like many people) am constantly learning about diverse perspectives and identities. If something was said in class (by anyone) that made you feel uncomfortable, please talk to me about it.

As a participant in course discussions, you should also strive to honor and respect the diversity of your classmates.

Title IX

The following is standard text provided by Dartmouth [here].

At Dartmouth, we value integrity, responsibility, and respect for the rights and interests of others, all central to our Principles of Community. We are dedicated to establishing and maintaining a safe and inclusive campus where all have equal access to the educational and employment opportunities Dartmouth offers. We strive to promote an environment of sexual respect, safety, and well-being. In its policies and standards, Dartmouth demonstrates unequivocally that sexual assault, gender-based harassment, domestic violence, dating violence, and stalking are not tolerated in our community.

The Sexual Respect Website at Dartmouth provides a wealth of information on your rights with regard to sexual respect and resources that are available to all in our community.

Please note that, as a faculty member, I am obligated to share disclosures regarding conduct under Title IX with Dartmouth's Title IX Coordinator. Confidential resources are also available, and include licensed medical or counseling professionals (e.g., a licensed psychologist), staff members of organizations recognized as rape crisis centers under state law (such as WISE), and ordained clergy (see https://dartgo.org/titleix_resourcesLinks to an external site.).

Should you have any questions, please feel free to contact Dartmouth's Title IX Coordinator or the Deputy Title IX Coordinator for the Guarini School. Their contact information can be found on the sexual respect website.