-
Notifications
You must be signed in to change notification settings - Fork 1
/
quickstart.tex
232 lines (188 loc) · 12.4 KB
/
quickstart.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
\documentclass[a4paper,11pt]{article}
\input{preamble}
\newcommand{\specrev}{\mbox{20181128-{\em draft}}}
\begin{document}
\title{\vspace{-0.7in}\Large {\bf FPGA-ZCU Quick Start Tutorial} \\
Document Version 2.0c
\vspace{-0.1in}}
\author{Tuo Li, Sri Parameswaran\\
School of CSE, University of New South Wales \\
{\tt [email protected], [email protected]} \\
\today
}
\date{}
\newpage
\maketitle
\newpage
\section*{About This Tutorial}
This document will provide a step-by-step walkthrough with the (rc-)fpga-zcu (the ZCU102 port of Rocket-chip) project, towards running operating system on rocket-chip System-on-Chip (SoC) on ZCU102 FPGA board.
This tutorial assumes using SD card to boot and program the FPGA.
This tutorial assumes that Vivado 2017.1 and SDK 2018.2 have been installed.
If you meet any problems, you might find the solution in Section~\ref{sec_errors}.
\newpage
\section{Getting Started}
\newcommand{\rootdir}{[root-repo-dir]}
\begin{enumerate}
\item Clone the main repository using git command from github:\\
{\tt git clone https://github.com/li3tuo4/rc-fpga-zcu.git}\\
Alternatively, you can directly download the zip file from github.
Let's denote the root directory of the main repo as \texttt{\rootdir}.
\item Enter the {\tt \rootdir/zcu102} directory and use Makefile to initiate the submodules/sub-repositories:\\
{\tt make init-submodules}
\item Enter {\tt \rootdir/rocket-chip} directory and follow README to install the rocket-chip dependencies (submodules). Make sure riscv-linux-gnu is installed when building compiler. This is required for making riscv-linux.
\end{enumerate}
\section{Hardware Generation}
\begin{enumerate}
\item Enter {\tt \rootdir/zcu102} directory and use Makefile to compile chisel source code:\\
{\tt make rocket}
\item Implement and build bitstream using Makefile:\\
{\tt make bitstream}
\item Export the hardware from the vivado workspace (the directory created during synthesis and implementation):
\begin{lstlisting}
cp zcu102_rocketchip_ZynqConfig/zcu102_rocketchip_ZynqConfig.runs/impl_1/rocketchip_wrapper.sysdef soft_config/rocketchip_wrapper.hdf
cp zcu102_rocketchip_ZynqConfig/zcu102_rocketchip_ZynqConfig.runs/impl_1/rocketchip_wrapper.bit soft_config/rocketchip_wrapper.bit
\end{lstlisting}
These files are used by software generation.
\end{enumerate}
\section{Software Generation}
In this step, we will create the software stacks for ARM and RISC-V. The ARM operating system will run on the host (ARM cores), which calls RISC-V via frontend server (fesvr).
Executing frontend server with Berkeley boot loader (bbl), built with RISC-V Linux kernel, on ARM, will boot up RISC-V core to run RISC-V Linux operating system.
NB!
If you only want to use riscv-pk (a lightweight proxy kernel) with fesvr, Step 1 to 5 in Section~\ref{sec:riscvsw} can be ignored. And, in Step 6 and 7, make and put pk into rootfs, instead of bbl.
\subsection{ARM}\label{sec-arm}
\begin{enumerate}
\item Install petalinux, which creates software stacks for ARM. Follow the ``Installation Steps'' on Page 11 in: \url{https://www.xilinx.com/support/documentation/sw_manuals/xilinx2017_1/ug1144-petalinux-tools-reference-guide.pdf}
\item Download board support package (bsp) file of ZCU102 from: \url{https://www.xilinx.com/member/forms/download/xef.html?filename=xilinx-zcu102-2017.1-final.bsp}. Xilinx account is required, which can be registered straightforwardly. Put the downloaded file {\tt xilinx-zcu102-2017.1-final.bsp} into {\tt \rootdir/zcu102/soft\_config} directory.
\item Export {\tt PETALINUX} environment variable and set up petalinux by:\\
{\tt export PETALINUX=/path/to/petalinux \&\& make pl\_setup}
\item Enter {\tt \rootdir/zcu102}. Use Makefile to automatically create and build petalinux project for ARM Linux image:\\
{\tt make kernel\_image}\\
Makefile will automatically create a petalinux project named {\tt petalinux\_proj}, in {\tt \rootdir/zcu102}.
The {\tt BOOT.BIN} and {\tt image.ub} in {\tt petalinux\_proj/images/linux} are the image files created. {\tt initramfs.cpio.gz} is the root file system.
\item Copy these three files to {\tt \rootdir/zcu102/fpga-images-zcu102} directory. Note this step will replace the initial image files in this directory. Note you can use Makefile to open, modify, and close the root filesystem:\\
{\tt make rootfs-open}\\
{\tt [modify]}\\
{\tt make rootfs-close}
\end{enumerate}
\subsection{RISC-V}\label{sec:riscvsw}
\begin{enumerate}
\item Make sure Xilinx SDK is installed and has been added to the PATH to successfully generate the binary.
Enter the zcu102 directory and use Makefile to create fesvr-zynq binary for ZCU102:\\
{\tt cd \rootdir/zcu102}\\
{\tt make fesvr-zynq}\\
The target files including \texttt{fesvr-zynq} and \texttt{libfesvr.so} will be generated in \texttt{\rootdir/common/build}.
\item Obtain freedom-u-sdk from \url{https://github.com/sifive/freedom-u-sdk}.
Make sure the system environment is set up as specified by README of the repo. Note you might need to unset \texttt{RISCV} environment variable for building riscv-linux (you can set it again after build). This tutorial uses {\tt linux\_u500vc707devkit\_confg} branch (commit {\tt db77bd7a8779d776d3f67b8a76ab5973e93604e6}).
\item Enter the freedom-u-sdk directory and initialize the repository:\\
\texttt{cd /path/to/freedom-u-sdk}\\
\texttt{git submodule update --recursive --init}
\item Copy the configuration file from {\tt \rootdir/zcu102/soft\_config} to the freedom-u-sdk (optionally you can backup the default config file at first):
\begin{lstlisting}
cp /path/to/freedom-u-sdk/conf/linux_defconfig /path/to/freedom-u-sdk/conf/linux_defconfig_bak
cp /path/to/soft_config/config_freedom /path/to/freedom-u-sdk/conf/linux_defconfig
\end{lstlisting}
\item In freedom-u-sdk root directory, make riscv-linux kernel:\\
{\tt make vmlinux}\\
The kernel image {\tt vmlinux} will be generated in \texttt{/path/to/freedom-u-sdk/work/linux}.
\item Follow the riscv-tools README to create Berkeley boot loader (bbl) with RISC-V Linux included:\\
{\tt cd \rootdir/rocket-chip/riscv-tools/riscv-pk/build\\
rm -rf *\\
../configure --prefix=\$RISCV --host=riscv64-unknown-linux-gnu --with-payload=/path/to/freedom-u-sdk/work/linux/vmlinux\\
make\\
make install}
\item Put {\tt fesvr-zynq, libfesvr.so} and {\tt bbl} into the root filesystem along with the commands introduced in Step 5 in Section~\ref{sec-arm}.
After you opened the initramfs (rootfs), copy the files into \texttt{/path/to/rootfs} (sudo probably is needed):
\begin{lstlisting}
cp /path/to/riscv-pk/build/bbl [root-repo-dir]/common/build/fesvr_zynq [root-repo-dir]/common/build/ /path/to/rootfs/home/root/
\end{lstlisting}
After you finish copying, close the rootfs.
\end{enumerate}
\section{FPGA Board Setup and SD Card Preparation}
\begin{enumerate}
\item Prepare the board following Xilinx quick start guide in \url{https://www.xilinx.com/support/documentation/boards_and_kits/zcu102/xtp426-zcu102-quickstart.pdf} and Chapter 2: Board Setup and Configuration in Xilinx UG1182 in \url{https://www.xilinx.com/support/documentation/boards_and_kits/zcu102/ug1182-zcu102-eval-bd.pdf}. Note Switch SW6 must be configured to ``off, off, off, on'' for allowing SD card boot up.
\item Prepare SD card following README in the sub-repository {\tt \rootdir/zcu102/fpga-images-zcu102} in \url{https://github.com/li3tuo4/fpga-images-zcu}.
\item Enter {\tt \rootdir/zcu102} directory and use Makefile to load SD card:\\
{\tt make load-sd}\\
Now it's ready to run the system on FPGA.
\end{enumerate}
\section{FPGA Emulation}
Make sure USB cable's driver is installed properly on the host PC. Xilinx UG344 might be helpful: \url{https://www.xilinx.com/support/documentation/user_guides/ug344.pdf}
\begin{enumerate}
\item Set up and open minicom or your preferred terminal:
{\tt minicom -D /dev/ttyUSB0}\\
Make sure the serial port in minicom is configured to ``115200 8N1'', which means ``speed = 115200, parity bit = none, data bits = 8''.
\item Insert the SD card and power on the FPGA board. The minicom terminal will show the ARM Linux boot up output. By default, ARM Linux's login username and password are both ``root''.
\item After login, execute fesvr with bbl as argument to wake up RISC-V and let it boot up Linux:
{\tt LD\_LIBRARY\_PATH=./ ./fesvr-zynq bbl}\\
By default, the login username and password of RISC-V Linux are ``root'' and ``sifive''.
\end{enumerate}
Hint: If you have ethernet connection, which is already available in the default system build, using SCP to transfer files such as fesvr and bbl directly from Host PC to FPGA after ARM Linux login can save the time in opening, modifying and closing root filesystem.
\section{Possible Build Errors and Solutions}\label{sec_errors}
\subsection{Error compiling sbt component `compiler-interface'}
Example error message after {\tt make rocket}:
\begin{lstlisting}
sajid2@sajid2-HP-Compaq-Elite-8300-SFF:~/fpga-zynq/zc706$ make rocket
mkdir -p /home/sajid2/fpga-zynq/common/build
cd /home/sajid2/fpga-zynq/common && java -Xmx2G -Xss8M -XX:MaxPermSize=256M -jar /home/sajid2/fpga-zynq/rocket-chip/sbt-launch.jar "run /home/sajid2/fpga-zynq/common/build zynq Top zynq ZynqFPGAConfig"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0
[ERROR] Failed to construct terminal; falling back to unsupported
java.lang.NumberFormatException: For input string: "0x100"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
.
.
.
.
.
[info] Resolving org.scala-sbt#interface;0.13.16 ...
[error] (compile:compileIncremental) Error compiling sbt component 'compiler-interface'
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? q
\end{lstlisting}
Solution:
\begin{enumerate}
\item Build/compile the chisel and verilator according to the instruction given here \url{https://github.com/freechipsproject/chisel3} including {\tt publish\_local} and adding librarydependencies into build.sbt file which locates into rocketchip directory.
According to the following instructions:\\
\begin{enumerate}
\item To publish your version of Chisel to the local Ivy (sbt's dependency manager) repository, run:
{\tt sbt publishLocal}
\item In order to have your projects use this version of Chisel, you should update the libraryDependencies setting in your project's build.sbt file to:
\begin{verbatim}
libraryDependencies += "edu.berkeley.cs" %% "chisel3" % "3.2-SNAPSHOT"
\end{verbatim}
\end{enumerate}
\item Install Java 8.
\begin{enumerate}
\item Downladed the java version 8 from \url{https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html}
\item Extract to the folder
\item Add the extracted folder bin path into path by
\begin{verbatim}export ......../bin :$PATH
\end{verbatim}
Check the java version before adding the path and after adding the path to confirm it. Use command {\tt which java}:
\begin{lstlisting}
sajid2@sajid2-HP-Compaq-Elite-8300-SFF:~/rc-fpga-zcu/rocket-chip/riscv-tools$ which java
/usr/bin/java
sajid2@sajid2-HP-Compaq-Elite-8300-SFF:~/rc-fpga-zcu/rocket-chip/riscv-tools$ export PATH=/home/sajid2/jdk-8u191-linux-x64/jdk1.8.0_191/bin:$PATH
sajid2@sajid2-HP-Compaq-Elite-8300-SFF:~/rc-fpga-zcu/rocket-chip/riscv-tools$ which java
/home/sajid2/jdk-8u191-linux-x64/jdk1.8.0_191/bin/java
\end{lstlisting}
\end{enumerate}
\item Now make rocket. Finally it is working now and successfully completing
\end{enumerate}
\subsection{tcmalloc failure during vivado synthesis}
Error message:
\begin{lstlisting}
---------------------------------------------------------------------------------
Start Technology Mapping
---------------------------------------------------------------------------------
src/central_freelist.cc:333] tcmalloc: allocation failed 16384
TclStackFree: incorrect freePtr. Call out of sequence?
[Tue Jan 15 17:23:56 2019] synth_1 finished
wait_on_run: Time (s): cpu = 00:00:00.33 ; elapsed = 00:20:41 . Memory (MB): peak = 1493.863 ; gain = 0.000 ; free physical = 1893 ; free virtual = 3117
# launch_runs impl_1 -to_step write_bitstream
ERROR: [Common 17-70] Application Exception: Failed to launch run 'impl_1' due to failures in the following run(s):
synth_1
These failed run(s) need to be reset prior to launching 'impl_1' again.
\end{lstlisting}
Solution:\\
It's caused by insufficient memory in the host machine.
According to \url{https://www.xilinx.com/products/design-tools/vivado/memory.html}, synthesizing for xczu9eg (zcu102) by vivado requires 10 to 14 GB memory.
\end{document}