xv6 Remote Debug using CLion

0.Introduction

This tutorial will teach you how to debug xv6 remotely using CLion. In the screenshot below, I have:
  • xv6 running remotely on a CS Lab machine
  • gdb connected to xv6 with frames and variables showed
  • code edited on the local machine and automatically synced
  • files on CS Lab machine listed
  • documentation (and declared location) shown for the selected function

If you haven't installed CLion or haven't set up file synchronization, you can read my previous tutorial: Setting up Coding Environment for CS 537. This tutorial is specifically focused on debugging xv6. Since xv6 is running in QEMU (instead of a host machine), we have to manually connect gdb to the gdb server, which complicates the setup process. I have another tutorial about debugging general C programs here: Remote Debug and Execution using CLion. Prof. Shivaram has a great tutorial about using gdb to debug xv6 during the discussion section. You can find the recorded video on this piazza page. I highly recommend to watch the video first, but this tutorial doesn't require any prior knowledge about gdb.

1. Preparation

Note: You can skip this part if you are compiling xv6 on your local computer or you are already in the CS Lab. 1. SSH to a CS Lab machine. Please use a specific server in the following list instead of best-linux.cs.wisc.edu. (best-linux.cs.wisc.edu is the Linux machine in the CS labs that currently has the fewest number of other users logged in to it. We don't want to use this because it might change over time.)
  • rockhopper-01.cs.wisc.edu through rockhopper-09.cs.wisc.edu
  • royal-01.cs.wisc.edu through royal-30.cs.wisc.edu
  • snares-01.cs.wisc.edu through anares-10.cs.wisc.edu
  • emperor-01.cs.wisc.edu through emperor-07.cs.wisc.edu
2. Copy xv6 to your working directory on a CS Lab machine, build the source, and run xv6 in debug mode.
cp -r /p/course/cs537-shivaram/xv6-sp19 .
make
make qemu-nox-gdb
3. Record the port number for gdb server. I always have 25784, but it might be different for you. (What qemu-nox-gdb does is that it runs a gdb server at localhost:25784 so that gdb can later connect to this server to debug the code.)

4. Open CLion and sync the folder to your local machine. Please refer to this tutorial if you don't know how to do this.

5. Open another terminal window, and use the following command to forward the port (recorded in step 2) to your local machine. (This is called SSH tunneling or SSH port forwarding so that we can access the gdb server running on the CS Lab machine via localhost:<port> on your local machine) Note: Please leave the terminal window open at the background in the following steps.
ssh -L <port>:localhost:<port> <username>@<server>.cs.wisc.edu

2. Configure CLion

1. Click "Edit configuration" on the top-right 2. Add new configuration of type "GDB Remote Debug" 3. Set 'target remote' args to be localhost:<port> 4. Choose the symbol file to be <local_folder>/kernel/kernel. (If you cannot find this file, build xv6 again and download the built folder to your local machine) 5. Set the system root to be <local_folder> 6. Add path mappings between <local_folder> and <remote_folder>

3. Run GDB Remote Debug

For testing, let's create a breakpoint at kernel/proc.c:261 by clicking the space next to the line number (same as "b kernel/proc.c:261" in gdb). Click the debug button on the top-right corner. You should be able to see a debug tab open on the bottom left, which shows you the frames (same as "info frame" in gdb) and variables (same as "info variables" and "print <var_name>" in gdb).

Here are some useful commands that you can use during debugging
  • Step over: execute next line of code. Will not enter functions. (same as "n" in gdb)
  • Step into: step to next line of code. Will step into a function. (same as "s" in gdb)
  • Step out: run until current stack frame finishes (same as "finish" in gdb)
  • Continue: continue running your program (same as "c" in gdb)
  • Pause: pause the execution of a running program (same as Ctrl+C in gdb)

To create a conditional breakpoint, right click on a normal breakpoint to add the condition.

You can also directly type commands into gdb by switching to the gdb tab (when the program is paused)

4. Some Useful Tips

For better code analysis and navigation, you can mark your xv6 folder as Project Sources and Headers, and add the line "include_directories(<relative_path_to_include_folder>)" to CMakeLists.txt

Once the previous step is done, you can press F1 on a function/variable to see the documentation. There is also an option for showing quick documentation on mouse move at "Editor - General - Other".

5. Reference

http://man7.org/linux/man-pages/man1/gdbserver.1.html https://sourceware.org/gdb/onlinedocs/gdb/Server.html https://www.jetbrains.com/help/clion/debugging-code.html https://www.jetbrains.com/clion/features/run-and-debug.html

Remote Debug and Execution using CLion

0. Introduction

I believed that most C programmers have encountered segmentation fault before. In this case, we have to write a lot of print statements or use the command-line tool gdb to find where is wrong. Either way, it's time-consuming to debug your code only using those basic tools. This tutorial walks you through how to debug on CLion locally while having your code compile and run on remote CS Lab machines. Note: If you haven't installed CLion before, you can read my previous tutorial here Permalink for this tutorial: https://shawnzhong.com/2019/02/10/tutorial-remote-debug-and-execution-using-clion/

1. Configure Remote Toolchain

A toolchain is a set of development tools including compiler, linker, debugger, etc. We want to use set up the remote toolchain for our project so that we can compile and run codes remotely. 1. Open setting and navigate to Build, Execution, Deployment > Toolchains 2. Click add at the bottom 3. Select Remote Host instead of System at the right of Name 4. Click open in the Credentials line 5. Fill in the information for the remote host

Use a specific server for the host (e.g. royal-10.cs.wisc.edu) instead of best-linux.cs.wisc.edu, since we don't want the project to be run on a different machine each time.

Here is a list of Linux server you can use:

  • rockhopper-01.cs.wisc.edu through rockhopper-09.cs.wisc.edu
  • royal-01.cs.wisc.edu through royal-30.cs.wisc.edu
  • snares-01.cs.wisc.edu through anares-10.cs.wisc.edu
  • emperor-01.cs.wisc.edu through emperor-07.cs.wisc.edu
6. If you have configured toolchain before, you need to drag the remote host to the top of the list to set it as the default toolchain.

Note 1: After you clicked OK, It may take some time for CLion to retrieve some information from the remote server. You can check the process by clicking the status information at the bottom of CLion. Please wait for that until you continue. Note 2: If cmake reports "CMake 3.13 or higher is required. You are running version 3.10.2", you need to change the first line of CMakeLists.txt to "cmake_minimum_required(VERSION 3.10.2)"

Now you should be able to compile and run your code remotely on CS Lab machine by clicking the run button at the top right corner of CLion.

2. Remote Debug on CS Lab Machine

Now you can click on the line number to add a breakpoint and click the debug button to debug your code. If you are encountering segmentation fault, the debugger will stop at the line where segfault happens.

3. Some Useful Tips

1. Make good use of the following three buttons on the debug tab.
  • Step over: go to the next line
  • Step into: look into the function call
  • Step out: get out of this function

2. You can add new watches to the variable list to check for a specific variable 3. Press F1 on a system function to show the documentation. You can also dock the documentation popup as a permanent tab on the right.

4. References

Setting up Coding Environment for CS 537

0. Introduction

As the projects get larger and more complicated, I think it is important to have a good coding environment for this course. I'd like to share with you how I use CLion as the main IDE for C programming. I recommend CLion for the following reasons:
  • Sync changed files to the remote server. You don't need to manually copy and paste the modified code to the CS lab.
  • Powerful code analysis and correction suggestion. You can eliminate most of the compilation errors while you are editing the file.
  • Built-in ssh and scp support with a handy graphical interface. For Windows users, you don't need to install PuTTY/WinSCP/WSL to connect to the CS lab.
  • Quick search and navigation. It's much easier to search files in the project folder (compared to grep) and navigate between the declaration and definition among files (compared to vim or emacs).
Please feel free to post any questions or comments down below :)

1. Installation

You can download CLion from this link: https://www.jetbrains.com/clion/. It is commercial software, but you can get the student license for free here: https://www.jetbrains.com/student/. The installer will ask you to configure toolchains for CLion. If you have not installed any toolchains  (gcc/llvm/MinGW/Cygwin) before, you can just skip this part, since we are probably not gonna to compile or run the code on our local machine. You can follow the default configuration to finish the installation. BTW, If you are a big fan of vim, you can enable the IdeaVim plugin on the last screen.

2. Connect to the Remote Server

After you created your first project, go to "Tools - Deployment - Configuration". Click the "+" button and select "SFTP" as the type. Under the connection tab, fill in the "Host", "User name", "Password" field, check the box for "Save password" (if you want to), and click "Autodetect" for "Root path". Under the mappings tab, choose your deployment path on the server (I put all files under /private/CS537. You can choose whatever path you want, but don't put it under /public) Now the remote server is configured. You can click "Tools - Start SSH session" to connect to the CS lab via ssh

3. Configure File Synchronization

If you select your project folder on the left and go to "Tools - Deployment", you can see the following menu. You can enable the "Automatic Upload" option so that local changes will be uploaded to the server automatically. You can also click "Browse Remote Host" to check the files on the CS lab. For setting up the xv6 files, I suggest to copy them from /p/course/cs537-shivaram/xv6-sp19 to your private remote folder (e.g. /u/s/z/szhong/private/CS537/p1b), and then click "Download from CS Lab". (Note that the deployment menu is also available if you right-click a certain folder or file so that you can manually upload/download a single file/folder)

4. Some Useful Tips

  • Press Control (on Windows) or Command (on Mac) to jump between definition and declaration. (It's pretty useful when navigating files in xv6)
  • Press Control + Shift + F to search among all project files
  • Double press shift to search for all file names/classes/actions...
  • Press Alt + Shift + L (on Windows) or Command + Shift + L (on Mac) to reformat code, or you can use the plugin called "Save Actions" to perform reformat on save
  • You can check the editing history under the menu "VSC - Local History - Show History".

Deploy WordPress on CS Department Server

Introduction

See https://csl.cs.wisc.edu/services/web

Download WordPress

1. Connect to CS Linux server

If you are using Mac or UNIX system, use the ssh command:

ssh [email protected]

If you are using Windows, download PuTTY to connect to the server

Note: Replace cs-username with your username

2. Change directory to ~/public/html

Type cd ~/public/html in the terminal, and press enter

Everything in this directory will be public at http://pages.cs.wisc.edu/~cs-username/

3. Download WordPress file and unzip it

Use wget to download the file from wordpress.org

wget https://wordpress.org/latest.zip

And use unzip to unzip the files

unzip latest.zip

4. Wordpress installed!

Open http://pages.cs.wisc.edu/~cs-username/wordpress/, and you should see the page below

Create a Database on GearHost

1. Create an account on GearHost

https://my.gearhost.com/account/signup

2. Create a database

Open the "Database" tab on the left

Click "Create Database" button

Choose a database name, and select MySQL (free)

Click "Create Empty Database"

Record your database name, database server, username, and password

Configure WordPress

1. Open http://pages.cs.wisc.edu/~cs-username/wordpress/ 2. Fill in the database information, and click submit

3. Copy the content of wp-config.php

Since we don't have the write permission to the server through webpage, we have to manually create this file

4. Go back to CS Linux server, and change directory to ~/public/html/wordpress

cd ~/public/html/wordpress

5. Open text editor, and create the file wp-config.php

nano wp-config.php

Note: You can also use vi or vim instead of nano, but nano is easier to use.

6. Paste the content

7. Control+X and press enter to save the file

8. Go back to the website and press install

9. WordPress installed!!!

Add Write Permission

1. Connect to CS Linux server and install "SSH SFTP Updater Support" plugin

ssh [email protected]

cd ~/public/html/wordpress/wp-content/plugins/

wget https://downloads.wordpress.org/plugin/ssh-sftp-updater-support.0.7.3.zip

unzip ssh-sftp-updater-support.0.7.3.zip

2. Enable the plugin on Dashboard

Open http://pages.cs.wisc.edu/~cs-username/wordpress/wp-admin/plugins.php

Click "activate" below "SSH SFTP Updater Support"

3. Configure "SSH SFTP Updater Support" Plugin

Open http://pages.cs.wisc.edu/~cs-username/wordpress/wp-admin/theme-install.php?browse=featured

Choose any theme you want, and click the install button

You will see a pop window called "Connection Information"

This is because we don't have permission to write file to the server through webpage

Choose SSH2 for connection type, since our CS server do not support FTP

Use best-linux.cs.wisc.edu for hostname, and your CS account for username and password.

Leave everything else blank, and click proceed.

Now you have the permission to write file to the server

Congratulations!

You have deployed WordPress on CS Linux Server!

# EOF