AFFINE Seminar: Week 0
For the next four weeks I’ll participate in the AFFINE seminar to study superintelligence alignment. During the seminar, I expect to pause my regular posts. Instead, I’ll send a brief weekly post summarizing what content was covered in the past week. I’ll then send a more thorough post after the seminar, collecting my thoughts and evaluating the full experience.
Why am I doing this?
I take seriously the conclusions of my own observations about artificial intelligence! I am bullish on the likelihood of continued AI advancement, and bearish on the likelihood of superintelligence alignment. Given my career background and personal aptitudes, I think I am well positioned to contribute somewhere in this problem space. Participating at AFFINE is an attempt to situate how I might do so.
Why AFFINE?
Individuals seeking to upskill on AI safety have a few basic routes they can pursue. I see four basic tracks, each with a representative program.
Foundations. Work at the forefront of AI research, typically focused on the mathematics underpinning artificial intelligence, neural networks, etc. An example program would be MATS, or possibly the new Illiad Intensive.
Engineering. Practical work focused on contemporary techniques for AI safety, such as machine interpretability. ARENA is a well-respected program here, operating out of LISA.
Alignment Theory. Theoretical work on the difficulties of alignment, leaning heavily on philosophy, decision theory, etc. Often focused on the anticipated challenges of AGI or ASI. AFFINE is an example of such a program. LensAcademy is another recent entry.
Policy. Work anticipating how society should deal with the consequences of AI progress, or how we can develop AI safely. You could also fit AI ethics into this category (including corporate ethics). I haven’t researched this space deeply, but I know BlueDot has a program. Some universities offer an online course.
Of these, I could have selected any of options 2 - 4. To approach foundations research in a serious way, I’d first need to do another program upskilling on the requisite math and computer science. In retrospect, attending a seminar like ARENA would have been nice for this reason, as it provides some of the basic experience with neural networks necessary for research.
However, it seems to me that alignment theory is a necessary starting point in approaching AI safety. AI safety is a very weird and nascent discipline. We really have no idea what techniques might be employed to develop superintelligence safely! If AI timelines are long (e.g., if reaching ASI takes 10+ years), it’s easy to imagine a future AI safety undergraduate/masters degree, which requires a course in alignment theory alongside a course in engineer safety, alongside a course in foundations, alongside a course in AI ethics! All these areas are complementary, and ultimately researchers and engineers should have strong intuitions in each. Unfortunately, timelines might be short, so we’re forced to make a decision somewhere, and alignment theory at least provides high optionality for future work.
This is the first year that the AFFINE seminar has run, so there is some inherent risk the seminar is a poor use of my time. While the seminar features a few well-known names in the safety community, there is no university accreditation for “superintelligence alignment,” meaning the quality of the seminar content is highly contingent on the individuals involved. As with any theoretical approach, the problems discussed may prove inapplicable to the real world. All my interactions with the seminar so far have been positive, but I recognize that I’m taking a higher-risk route toward AI safety than, for example, ARENA.
What’s next?
For the next four weeks I will be holed up in a chateau in the Czech countryside with approximately 40 other individuals from a wide variety of backgrounds (mostly CS) learning the theory behind superintelligence alignment. I expect to spend a lot of time reading course materials, participating in discussions, meeting with safety researchers, hopefully doing a bit of supplementary research on neural network architecture, and taking a couple hikes.
My posts during the seminar will be a bit less polished than usual—closer to a journal entry, or a letter from an estranged friend who developed a psychosis then fell into a cult. If you have any questions you’d like me to bring to the seminar, please let me know in the comments!
