Tag Archives: github

Join us to discuss using AI to solve simple programming problems on Monday 3rd April at 2pm BST

Posted on by
CC licensed pilot icon from flaticon.com

Maybe you wrote that code and maybe you didn’t. If AI helped you, such as the OpenAI Codex in GitHub Copilot, how did it solve your problem? How much did Artificial Intelligence help or hinder your solution? Join us to discuss a paper by Michel Wermelinger from the Open University published in the SIGCSE technical symposium earlier this month on this very topic. [1] We’ll be joined by Michel who will present a lightning talk to kick-off our discussion. Here’s the abstract of his paper:

The teaching and assessment of introductory programming involves writing code that solves a problem described by text. Previous research found that OpenAI’s Codex, a natural language machine learning model trained on billions of lines of code, performs well on many programming problems, often generating correct and readable Python code. GitHub’s version of Codex, Copilot, is freely available to students. This raises pedagogic and academic integrity concerns. Educators need to know what Copilot is capable of, in order to adapt their teaching to AI-powered programming assistants. Previous research evaluated the most performant Codex model quantitatively, e.g. how many problems have at least one correct suggestion that passes all tests. Here I evaluate Copilot instead, to see if and how it differs from Codex, and look qualitatively at the generated suggestions, to understand the limitations of Copilot. I also report on the experience of using Copilot for other activities asked of students in programming courses: explaining code, generating tests and fixing bugs. The paper concludes with a discussion of the implications of the observed capabilities for the teaching of programming.

All welcome, as usual we’ll be meeting on zoom, details at sigcse.cs.manchester.ac.uk/join-us. 

References

  1. Michel Wermelinger (2023) Using GitHub Copilot to Solve Simple Programming Problems in Proceedings of the 54th ACM Technical Symposium on Computer Science Education Pages SIGCSE 2023 page 172–178 DOI: 10.1145/3545945.3569830

Join us to discuss Collaborative Coding in the Cloud on Monday 6th February at 2pm GMT

Posted on by
Reply
Creative Commons cloud image by flaticon.com

More and more software development tools are available in the cloud, with tools like Replit, CodingRooms, GitHub Codespaces, Amazon Web Services Cloud9, JetBrains and Eclipse all offering online tools for developers to code collaboratively in the cloud. Integrated Development Environments (IDEs) which have traditionally been available as “fatter” clients are increasingly available as “thinner” web-based clients running in a browser. These tools can lower some of the barriers to installation and maintenance for their users. What are the strengths and weaknesses of these new tools for teaching introductory programming courses? Join us on Monday 6th February at 2pm GMT to discuss a paper by Phil Hackett and his colleagues at the Open University on this very topic [1], from the abstract:

This paper discusses a pilot research project, which investigated the use of online collaborative IDEs (Integrated development environments) during a first-year computing degree course. The IDEs used can be described as virtual computing labs because they replicate some of the actions possible in physical computing labs. Students were supported by a tutor with real-time help and feedback provided, whilst they were programming, without being collocated. The use of two different platforms is considered with the benefits and drawbacks discussed. Students and tutors indicated that they would like to use a virtual computing lab approach in the future.

We’ll be joined by the lead author of the paper Phil Hackett, who’ll give us a lightning talk summary of the paper to kick-off our journal club discussion. The paper was presented at Computing Education Practice (CEP) in Durham earlier this month. [1]

All welcome, as usual we’ll be meeting on zoom, details at sigcse.cs.manchester.ac.uk/join-us. 

References

  1. Phil Hackett, Michel Wermelinger, Karen Kear and Chris Douce (2023) Using a Virtual Computing Lab to Teach Programming at a Distance in CEP ’23: Proceedings of 7th Conference on Computing Education Practice Pages 5–8 DOI:10.1145/3573260.3573262

Join us to discuss the implications of the Open AI codex on introductory programming Monday 4th July at 2pm BST

Posted on by
Reply
Robot icon by Freepik – Flaticon


Automatic code generators have been with us a while, but how do modern AI powered bots perform on introductory programming assignments? Join us to discuss the implications of the OpenAI Codex on introductory programming courses on Monday 4th July at 2pm BST. We’ll be discussing a paper by James Finnie-Ansley, Paul Denny, Brett A. Becker, Andrew Luxton-Reilly and James Prather [1] for our monthly SIGCSE journal club meetup on zoom. Here is the abstract:

Recent advances in artificial intelligence have been driven by an exponential growth in digitised data. Natural language processing, in particular, has been transformed by machine learning models such as OpenAI’s GPT-3 which generates human-like text so realistic that its developers have warned of the dangers of its misuse. In recent months OpenAI released Codex, a new deep learning model trained on Python code from more than 50 million GitHub repositories. Provided with a natural language description of a programming problem as input, Codex generates solution code as output. It can also explain (in English) input code, translate code between programming languages, and more. In this work, we explore how Codex performs on typical introductory programming problems. We report its performance on real questions taken from introductory programming exams and compare it to results from students who took these same exams under normal conditions, demonstrating that Codex outscores most students. We then explore how Codex handles subtle variations in problem wording using several published variants of the well-known “Rainfall Problem” along with one unpublished variant we have used in our teaching. We find the model passes many test cases for all variants. We also explore how much variation there is in the Codex generated solutions, observing that an identical input prompt frequently leads to very different solutions in terms of algorithmic approach and code length. Finally, we discuss the implications that such technology will have for computing education as it continues to evolve, including both challenges and opportunities. (see accompanying slides and sigarch.org/coping-with-copilot/)

All welcome, details at sigcse.cs.manchester.ac.uk/join-us. Thanks to Jim Paterson at Glasgow Caledonian University for nominating this months paper.

References

  1. James Finnie-Ansley, Paul Denny, Brett A. Becker, Andrew Luxton-Reilly, James Prather (2022) The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming ACE ’22: Australasian Computing Education Conference Pages 10–19 DOI:10.1145/3511861.3511863

Join us to discuss learning git on Monday 5th October at 2pm

Posted on by
Reply
Git Logo by Jason Long at git-scm.com/downloads/logos is licensed under a Creative Commons Attribution 3.0 Unported License.

The use of git is widespread in software engineering, however many novices struggle to get to grips with its complex distributed information model, challenging command line syntax and leaky abstractions. To investigate these pitfalls, we’ll be talking about a paper published by Santiago Perez De Rosso and Daniel Jackson on Purposes, Concepts, Misfits, and a Redesign of Git at OOPSLA. [1] From the abstract:

Git is a widely used version control system that is powerful but complicated. Its complexity may not be an inevitable consequence of its power but rather evidence of flaws in its design. To explore this hypothesis, we analysed the design of Git using a theory that identifies concepts, purposes, and misfits. Some well-known difficulties with Git are described, and explained as misfits in which underlying concepts fail to meet their intended purpose. Based on this analysis, we designed a reworking of Git (called Gitless) that attempts to remedy these flaws.

To correlate misfits with issues reported by users, we conducted a study of Stack Overflow questions. And to determine whether users experienced fewer complications using Gitless in place of Git, we conducted a small user study. Results suggest our approach can be profitable in identifying, analysing, and fixing design problems.

So what’s wrong with git?

Santiago’s presentation on What’s Wrong With Git? at Git Merge in 2017

Details of the zoom meeting have been posted on our slack workspace, see sigcse.cs.manchester.ac.uk/join-us for further information. Thanks to Juha Sorva at Aalto University for recommending this paper.

Journal club dates for your diary

We’ll be meeting on the first Monday of every month throughout autumn, so if you’d like to join us next month or a subsequent month, add these journal club dates to your diary:

  • Monday 5th October at 2pm
  • Monday 2nd November at 2pm
  • Monday 7th December at 2pm

References

  1. Santiago Perez De Rosso and Daniel Jackson (2016) Purposes, Concepts, Misfits, and a Redesign of Git in Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, (OOPSLA), pages 292–310 DOI: 10.1145/2983990.2984018