Mass Import GitHub Issues

Recently we had to move one repository to a different org. Discovered that GitHub can fork all source code but issues list is not included. I thought to document this process because it may be useful at some point later.

I did usual searches and there are several options and open source utils written to import issues. However, all of them required extensive local setup, config changes, installing various dependencies with some base language or runtimes, scripts etc. This is not always possible for security reasons some of the dependencies are not allowed etc. One simple method was to use GitHub CLI gh.

The GitHub CLI has many useful commands, but no exact issue export/import. Instead there are useful issue commands:

gh issue list
gh issue create

Export Issues

There is no direct export command. The CLI gh provides issue list command that can output issues in JSON format.

gh issue list command requires number of options to make output useful for migration. Need to set following items:

limit - number of issues to export some reasonable value 1000
state - to get only open or other types of issues
json - option to output JSON, need to also set what fields to include.

Here is an example command to output only open issues, note that we also redirect output to a file to be use in next step.

gh issue list --limit 1000 --state open --json 'number,title,body,state,createdAt,updatedAt,label' > ../issues.json

The result data in JSON format will look like this example bellow.

[
  {
    "body": "**OVERVIEW**\r\n\r\nThe following is from a list from the Chicago Sprint on Improvements that the group felt would be important to users:\r\n\r\n**The actually word count is undercounting by 1 **\r\n\r\nhttps://salesforce.quip.com/JKMnAsPUEeDd\r\n\r\n\r\n\r\n**ACCEPTANCE CRITERIA**\r\n\r\n**OUT OF SCOPE**\r\n\r\n**BACKGROUND/REFERENCE**\r\n",
    "createdAt": "2023-12-13T23:19:46Z",
    "labels": [],
    "number": 120,
    "state": "OPEN",
    "title": "UI Improvement - word count inaccurate",
    "updatedAt": "2023-12-13T23:19:46Z"
  },
  {
    "body": "**OVERVIEW**\r\n\r\nThe following is from a list from the Chicago Sprint on Improvements that the group felt would be important to users:\r\n\r\n**Be able to clone an existing grant application to use with another grant**\r\n\r\nhttps://salesforce.quip.com/JKMnAsPUEeDd\r\n\r\n\r\n**ACCEPTANCE CRITERIA**\r\n\r\n**OUT OF SCOPE**\r\n\r\n**BACKGROUND/REFERENCE**\r\n",
    "createdAt": "2023-12-13T23:17:49Z",
    "labels": [],
    "number": 119,
    "state": "OPEN",
    "title": "UI Improvement - Cloning Grant Proposals",
    "updatedAt": "2023-12-13T23:17:49Z"
  },
  {
    "body": "**OVERVIEW**\r\n\r\nThe following is from a list from the Chicago Sprint on Improvements that the group felt would be important to users:\r\n\r\n**Be able to see the Account preview to confirm it's the correct account**\r\n\r\nhttps://salesforce.quip.com/JKMnAsPUEeDd\r\n\r\n\r\n**ACCEPTANCE CRITERIA**\r\n\r\n**OUT OF SCOPE**\r\n\r\n**BACKGROUND/REFERENCE**\r\n",
    "createdAt": "2023-12-13T23:16:43Z",
    "labels": [],
    "number": 118,
    "state": "OPEN",
    "title": "UI Improvement - Relate Account Info",
    "updatedAt": "2023-12-13T23:16:43Z"
  }
]

Convert JSON File to list

JSON file with all relevant issues we need to import needs to be transformed into some list so we can import/create issues programatically.

GitHub does not provide any easy import file feature so we can use CLI gh issue create command to add new issues to new repository. We will create a command script that will list all issue create commands to do bulk create operation.

To convert JSON file into this command list I have chosen to write a small and simple NodeJS script to read in our issues JSON file and output list of gh create commands.

const { readFileSync } = require('fs');

let data = readFileSync('./issues-prety.json');

const jslist = JSON.parse(data);
jslist.forEach(function(table) {
    let title = table.title;
    let body = table.body.replace(/(?:\r\n|\r|\n)/g,'<br>');
    if (table.labels.length > 0){
    	const lbl = table.labels[0].name;
    	console.log(`gh issue create --title "${title}" --body "${body}" --label "${lbl}"`);
    }else{
    	console.log(`gh issue create --title "${title}" --body "${body}"`);
    }
    console.log(`# --- create issue ---`);
});

Note that exported issues file has some \r\n new line tags. Using REGX we can clean this and replace new line with markdown to correctly format new issue text. This code line: let body = table.body.replace(/(?:\r\n|\r|\n)/g,'<br>');

Now the result script has a clean list of all issues we can create them in new repository. Our command list will look like this sample and may have many more commands to create as many issues as needed.

gh issue create --title "UI Improvement - word count inaccurate" --body "**OVERVIEW**<br><br>The following is from a list from the Chicago Sprint on Improvements that the group felt would be important to users:<br><br>**The actually word count is undercounting by 1 **<br><br>https://salesforce.quip.com/JKMnAsPUEeDd<br><br><br><br>**ACCEPTANCE CRITERIA**<br><br>**OUT OF SCOPE**<br><br>**BACKGROUND/REFERENCE**<br>"
# --- create issue ---
gh issue create --title "UI Improvement - Cloning Grant Proposals" --body "**OVERVIEW**<br><br>The following is from a list from the Chicago Sprint on Improvements that the group felt would be important to users:<br><br>**Be able to clone an existing grant application to use with another grant**<br><br>https://salesforce.quip.com/JKMnAsPUEeDd<br><br><br>**ACCEPTANCE CRITERIA**<br><br>**OUT OF SCOPE**<br><br>**BACKGROUND/REFERENCE**<br>"
# --- create issue ---
gh issue create --title "UI Improvement - Relate Account Info" --body "**OVERVIEW**<br><br>The following is from a list from the Chicago Sprint on Improvements that the group felt would be important to users:<br><br>**Be able to see the Account preview to confirm it's the correct account**<br><br>https://salesforce.quip.com/JKMnAsPUEeDd<br><br><br>**ACCEPTANCE CRITERIA**<br><br>**OUT OF SCOPE**<br><br>**BACKGROUND/REFERENCE**<br>"
# --- create issue ---
gh issue create --title "UI Improvement - Preview function" --body "**OVERVIEW**<br><br>The following is from a list from the Chicago Sprint on Improvements that the group felt would be important to users:<br><br>**A preview function (maybe that’s just leveraging the HTML) or like Igor suggested, the preview would have a copy button next to each section.**<br><br>https://salesforce.quip.com/JKMnAsPUEeDd<br><br><br>**ACCEPTANCE CRITERIA**<br><br>**OUT OF SCOPE**<br><br>**BACKGROUND/REFERENCE**<br>"
# --- create issue ---
gh issue create --title "UI Improvement - Caching Bugs" --body "**OVERVIEW**<br><br>The following is from a list from the Chicago Sprint on Improvements that the group felt would be important to users:<br><br>**Reorder delay /the visualized order numbers are not updated immediately = bugs fixed**<br><br>https://salesforce.quip.com/JKMnAsPUEeDd<br><br>**ACCEPTANCE CRITERIA**<br><br>**OUT OF SCOPE**<br><br>**BACKGROUND/REFERENCE**<br>"

Create List of issues

In order to execute CLI gh commands on new repository and org CLI must be authenticated to this new GitHub Org. Change directory to new repository cloned locally and run our CLI command. Final result, we migrated a long list of issues from one repository to a new Org repository.

Known Issues

Some of the issue text may contain some characters like " that will need escape or replacements to avoid breaking command execution.

The issue numbers are not migrated from origin because we can only create as new issues and GitHub will assign new issue numbers.