Computational Photography (FA20)

Overview

Note! The syllabus is still in flux and subject to adjustment as the quarter progresses.

 

Note! All times listed on this page are EDT (Dartmouth's local time).

Computational photography lies at the intersection of photography, computer vision, image processing, and computer graphics. At it’s essence, it is about leveraging the power of digital computation to overcome limitations of traditional photography. In this course, we will study the increasing computational aspects of digital photography, with an emphasis on software techniques.

The first half of the term will be lecture/assignment based. We'll learn how a camera's optics form images on a sensor, how to represent images digitally on a computer, how humans perceive colors, how cameras capture color, and how we can represent color mathematically. You will build a simple camera from scratch and implement various image processing algorithms for photography, including high-dynamic-range photography and tone mapping, image morphing, and advanced image editing algorithms. This is not a photography class or a class on Photoshop/Lightroom. However, you will learn the math and algorithms behind popular image manipulation and processing tools like Adobe Photoshop and Lightroom. Course work will include taking some photos and implementing several algorithms to manipulate those photos in weekly programming assignments (using C++).

In the second half of the term we will all read, present and discuss foundational and recent research papers in the field of computational imaging. Each student will also work on a final project.

By the end of the course, you should have an understanding of the capabilities and limitations of photography today, and have sufficient background to implement new solutions to photography challenges.

Course staff

Instructor: Wojciech Jarosz
TA: Shaojie Jiao

Lectures

MWF, 1:10–2:15 pm
Location: Zoom

X-hour

Tu, 1:40–2:30 pm
Location: Zoom

Office Hours

Professor:
W, 4:15pm–5pm
Th, 9:15am-10am
Location: Zoom

TA:
M, 6:00pm-6:45pm
F, 10:45am-11:30am
Location: Zoom

Prerequisites

Successful completion of COSC 10 and COSC 70 (formerly 70.01 and 11) are required. We will be using C++ for the programming assignments. You should either already be familiar with C++, or be able to pick it up quickly from knowledge of related languages. With knowledge of Python (COSC 1), Java (COSC 10), and C (COSC 50, recommended) this should be straightforward.

Equipment

Some assignments will require taking a few photos. You will need access to a camera with manual exposure control. Any digital camera with manual controls should work, including most smartphones from the last 5 years with an appropriate camera app. If you have a dSLR, great (and have fun!), but it is not required.

There is no required textbook. Lecture material will be posted on Canvas.

Live Lectures/Discussions, and our online format

We will conduct all lectures live over zoom during our regularly scheduled class time (the zoom link is above). My expectation is that you all try to join these live if you can. I will also record our lectures so that the few of you who cannot join at the regular time can still watch the lectures, and so that you can all use the recordings as reference material. I will also post the corresponding slides in the Files section. We will also use this zoom for our live paper discussions in the second half of the term. 

Make sure to read the info on zoom etiquette and consent to recording below.

Email/communication policy

Instead of email, we will be using Slack for class communication. You can join our Slack by clicking on the appropriate section in Canvas' navigation after which point you can use the web, mobile or desktop client.

Upon joining, you'll be added automatically to a number of channels.

We'll post general class announcements in the #announcements channel.

There is a #general channel which you should use for general Q&A and discussion about course content. 

We've also created #assignmentN channels for assignment-specific questions and discussion.

We may add a some more channels as the term progress, and when we do, you'll need to join those manually.

DMs and Email

Please do not email or direct message the course staff individually with question about assignments, etc. We want to make sure everyone benefits from answers to each question, and encourage you to also collectively discuss and answer each others' questions. The only exception to this is if your message is of a personal or sensitive nature (don't post portions of your code). In this case, please send it by email. I will not check or respond to unsolicited direct messages.

Coursework & Grading

The tentative grading scheme will be:

  • 60%: Weekly assignments (mostly C++ programming)
  • 20%: Class engagement (attendance, paper reading, discussion, and presentation)
  • 20%: Final project
Disclaimer: The grade breakdown is subject to adjustments.

There will be no quizzes, tests, midterms, or a final in this class.

Please check the Assignments section in Canvas for more details on the above components.

Graduate & Extra Credit

Grads and undergrads enrolled in the class will be graded separately and we will generally grade grads more strictly. The programming assignments will typically include extra portions that will be required for students enrolled in the graduate version and may count as extra credit for those enrolled as undergrads. We will grade these separately and grant extra credit only if the foundation of the original assignment has been completed correctly (e.g., at least 90% of the original requirements/grade are met).

Extra credit will not affect the grading curve. At the end of the course, extra credit will not be added directly to your score. Instead, we will consider your extra credit to potentially round your grade more favorably when discretizing to letter grades.

Late Submissions

Each assignment will be submitted through Canvas, and will have a strict deadline. The assignments will build off of each other, and sometimes the basecode for the next assignment will include solutions to portions of the previous assignment. Due in part to this, we will not accept late submissions for programming assignments (e.g. an assignment handed in just 5 minutes late will get 0 credit).

Exception to this rule might be made for special cases only if you inform me (Prof. Jarosz) well (e.g. >1 week, not a couple days) before the deadline, or in the case of a medical emergency, and at my sole discretion.

It’s up to you to check that assignments have been successfully submitted to Canvas; if there are any problems, you must notify course staff by email immediately (not weeks later, claiming “Canvas ate it”).

Assignment regrading requests must be submitted via email to the course staff within one week of the original grade posting.

Honor principle – Not your standard warning!

(This section is adapted from Tim Tregubov's text from CS 52 Spring 2020).

The web is an open place and there is a lot of code out there. We want you to be able to show your work and to be proud of the code you have written. However, you do need to learn and as such all the code for the assignments and for your portion of the final project will need to actually be written by you.

The Internet

The Internet will be a large source of information and help to you as you do your assignments in this class. We want to encourage you to scour the internet / stackoverflow for help, but you must credit any solutions you find in comments in your code specifying where you found them and what you took from them. If you copy/paste code without attribution you are violating the Academic Honor Principle, and we hope you enjoy your 3 term vacation. You will not be penalized for copied code as long as it is properly attributed and some significant portion of your project was still written by you. Obviously don’t copy/paste the whole project.

Student Collaboration

For the programming assignments, you may verbally collaborate as much as you want. However, the code that you turn in, should be created, typed, documented, and output generated, by you. You may consult freely with instructors and classmates but you should work individually when actually coding. If you copy or look at code from another current or previous student for “debugging” purposes, it will be considered a violation of the Academic Honor Principle unless you cite that you have done so in the code (and your readme file). All collaboration with another student should be documented in the comments and cited, exactly as you would with an internet resource. If there is any question about whether you wrote the code yourself or were inspired by or copied off of others, just CITE IT. You may lose some points for not implementing that specific component yourself, but properly cited code will not be considered a violation of the Academic Honor Principle.

Citation Format

Make your in-code citations look something like this (policy adapted from MIT Integrity Handbook):

// :information_source: adapted from: http://stackoverflow.com/someurl
// this method does x and y

or

// :information_source: copied off of Cameron Howe's code for this assignment
// this method does x and y

Importantly, note the use of the ":information_source:" emoji string.

Additionally, you will submit a readme.md file with each assignment where you should copy all the relevant citations into a "Bibliography" section:

# Assignment 1

* Some bullets
  * a subbullet
  
## Bibliography

In completing this assignment, I used these sources:

:information_source: adapted from: http://stackoverflow.com/someurl
this method does x and y

:information_source: copied off of Cameron Howe's code for this assignment
this method does x and y

Honor Code Violation Examples

These are designed to serve to help illustrate the various situations that might arise and whether they are or are not violations of the Academic Honor Code. I’m required to report any such case to the Dartmouth Office of Judicial Affairs, resulting in a lot of wasted time, a COS hearing, and possible suspension or even expulsion from the College. Regardless of the outcome of the COS hearing, I reserve the right to assign a failing grade on the assignment where we suspect an honor code violation occurred, or the entire course.

  • You find a block of code online that you don’t quite understand but it seems to work either when you make your code look similar or you copy/paste it in. Either because you are embarrassed or careless you do not cite where you got the code from. This is a violation of the Honor Code. CITE IT and you will be fine.
  • It is midnight and the assignment is due and your friend sends you a block of code to help debug. You try to not look at it, but you end up turning in something very similar to what your friend sent. This would be attribution error and an Academic Honor Code violation if you did not cite it. CITE IT LIKE YOUR LIFE DEPENDS ON IT. If you had turned it in with an attribution: “I whole-heartedly copied this from my friend N”, we would deduct points from your assignment, but that is it!
  • You and a couple of friends are working together on an individual (not team) assignment. You are talking about it and planning it out on the whiteboard. If you are using natural spoken language you are ok. If you are writing code on the board or telling each other what to type in, that is a violation of the Academic Honor Code if you do not cite it. If you have not properly attributed work to your friend and you end up with the same code you would face a COS hearing to disentangle what happened. These hearings are brutal with back-stabbings, tears, and ruined friendships. Why risk it? The best approach when talking about your assignments together is to do it as if you were helping each other learn: ask questions, give hints. That is why we are all here after all. If you type in what someone tells you to type, CITE IT.
  • In most assignments you will be asked to also submit the output of your code (typically an image). If that output does not originate from the code you submit, this is a violation of the Academic Honor Code if you do not clearly mark it as coming from a different source, which you MUST CITE.
  • You leave your code on a computer or server, or posted it online, and someone else can see it, then they can copy it and hand it in. It’s often difficult to tell who was the copy-ee and who was the copy-er (unless you fell victim to the rare breed of ethical copy-er who clearly cited it with something like ":information_source: I saw this on Nancy's computer screen"). Avoid this situation.

But officer I panicked!

If you are stuck: ask for an extension, come to office hours, email your Professor and your TA, learn some humility and accept that you might need help. Asking for help is one of the greatest skills that one can possess, and if you get anything from this class know that it is OK to ask for help.

Please do not create a horrible situation for all of us by copying code without citing the source. Our job is to help you learn. You will not be penalized for asking.

Zoom etiquette

By now you probably all have Zoom installed (if not get it from dartmouth.zoom.us, the web version has fewer features). Sign in using your Dartmouth credentials. Play with it before you come to class. Learn how to mute yourself, pause your video, raise your hand, etc.

Here are a few things to keep in mind:

  1. Please find a quiet place, or plan to use a headset with an integrated mic. If you can’t, you’ll still be able to ask questions by typing them into chat.

  2. Keep yourself on mute until called on. (You can unmute briefly by holding down the space bar.) If you have a question, you can raise your hand to get my attention using the participant menu.

  3. Use your full name (no 31337 H4X0R handles please) when you sign on – it will help me learn your names and facilitates discussion.
  4. It’s nice to use video to display your face. Please make sure your background and attire are something you and the class will be comfortable with. A good rule of thumb is to wear whatever you would wear to a lecture at Dartmouth.

  5. I understand you may periodically need to turn off your video or step away. For those cases, please set up your zoom profile with a photo (of your face) so that when you do turn off your video we can still see you.

  6. I know it can be difficult, but please try to be completely engaged when attending a Zoom lecture or Zoom office hours (no web browsing or social media).

  7. Please be polite and kind to everyone.

I’ll be recording all of our lectures that occur at the scheduled time, so that those who cannot be there due to poor internet connectivity or time zones can watch later. Dartmouth has asked that I include the following language describing some bounds on how recording should be used. You do not need to send me any sort of agreement on this – it just makes clear that you shouldn’t record or distribute any recordings without my consent.

Consent to recording of course and group office hours

  • I affirm my understanding that this course and any associated group meetings involving students and the instructor, including but not limited to scheduled and ad hoc office hours and other consultations, may be recorded within any digital platform used to offer remote instruction for this course;

  • I further affirm that the instructor owns the copyright to their instructional materials, of which these recordings constitute a part, and distribution of any of these recordings in whole or in part without prior written consent of the instructor may be subject to discipline by Dartmouth up to and including expulsion;

  • I authorize Dartmouth and anyone acting on behalf of Dartmouth to record my participation and appearance in any medium, and to use my name, likeness, and voice in connection with such recording; and

  • I authorize Dartmouth and anyone acting on behalf of Dartmouth to use, reproduce, or distribute such recording without restrictions or limitation for any educational purpose deemed appropriate by Dartmouth and anyone acting on behalf of Dartmouth.

Requirement of consent to one-on-one recordings.

By enrolling in this course, I hereby affirm that I will not under any circumstance make a recording in any medium of any one-on-one meeting with the instructor without obtaining the prior written consent of all those participating, and I understand that if I violate this prohibition, I will be subject to discipline by Dartmouth up to and including expulsion, as well as any other civil or criminal penalties under applicable law.

Accessibility Needs

Let me know before the end of the second week of the term if you will need a disability-related accommodation or service. In order for accommodations to be authorized, students are required to consult with Student Accessibility Services ("SAS" <student.accessibility.services@dartmouth.edu>; SAS website; phone: 603-646-9900) and to email me their SAS accommodation form. We will then work together with SAS if accommodations need to be modified based on the online learning environment. If students have questions about whether they are eligible for accommodations, they should contact the SAS office. All inquiries and discussions will remain confidential.

Wellness

We recognize that the academic environment at Dartmouth is challenging, that our terms are intensive, and that classes are not the only demanding part of your life. There are a number of resources available to you on campus to support your wellness, including: your undergraduate dean, Counseling and Human Development, and the Student Wellness Center. I encourage you to use these resources and come speak with me to take care of yourself throughout the term.

COVID-19 addendum

While the COVID-19 pandemic has already changed how this course is structured, it has the potential to result in further personal impact which may prevent you from continuing engagement in the class. This may be due to contraction of the disease by you or a loved one, increased familial responsibilities, financial difficulties, or impacts on your mental/emotional health.

I have structured the course so that, hopefully, these disruptions will not prevent you from successfully learning the material.

In the event that you are directly or indirectly impacted by COVID-19 in such a way that will affect your performance in the course, it is imperative that you reach out to me as soon as possible. You may also reach out to your undergraduate Dean if that would make you more comfortable. We cannot assist you if we don’t know there is a problem. Our first priority is your health and security. We will work to put you in touch with appropriate resources to assist you.

Acknowledgements

The materials from this class borrow heavily from slides and assignments prepared by other instructors. In particular, many materials are modified from those of Frédo Durand, Steve Marschner, Matthias Zwicker, Alexei A. Efros, Marc Levoy, and others, as noted in the slides and assignments. If you reuse this material for academic or research purposes, please maintain all acknowledgments.

Course Summary:

Date Details Due