This week, we will go into detail about pixels and zeros and ones that make up images. In particular, we will be going deeper into the fundamental building blocks that make up files.
Further, we will discuss how to access underlying data stored in computer memory.
Pixels are the smallest addressable elements in an image that are arranged on an up-down, left-right grid.
We can imagine an image as a map of bits, where zeros represent black and ones represent white.
Is a color model in which numbers representing the amount of red, green and blue primary colors are added together to reproduce a color.
In this photoshop color selector, the amount of red, green and blue will change the color selected.
Notice the notation
#0000FF
representing the color blue.
R: 0
means no amount of red, and is represented by the first two digits00....
G: 0
means no amount of green, and is represented by the next two digits..00..
B: 255
means maximum amount of blue, and is represented by the last two digits....FF
Hexadecimal or base-16 is a numeral system that represents numbers using 16 counting values:
0 1 2 3 4 5 6 7 8 9 a b c d e f
When counting in hexadecimal, each column is a power of 16
.
| 16^1 | 16^0 | = | 16 | 1 |
| # | # | | 0 | 0 |
Number | Hexadecimal [16 - 1] |
---|---|
0 | 00 |
1 | 01 |
9 | 09 |
10 | 0A |
15 | 0F |
16 | 10 |
255 | FF |
Important
255 is the highest number you can count using two-digit hexadecimal system.
Hexadecimal is useful because you can represent 4 bits using a single digit or 8 bits (1 byte) using just two digits:
F = 1111
FF = 11111111
In hexadecimal notation, number 16 is represented as 10
, number 17 is represented as 11
and number 25 is represented as 19
.
You can imagine how confusing this can be. To clarify things, by convention, all hexadecimal numbers are ofter represented with the 0x
prefix.
0x0 0x1 0x2 0x3 0x4 0x5 0x6 0x7 0x8 0x9 0xA 0xB 0xC 0xD 0xE 0xF 0x10 0x11
0x12 0x13 0x14 0x15 0x16 0x17 0x18 0x19 0x1A 0x1B 0x1C 0x1D 0x1E 0x1F 0x20
Let's explore this in code:
#include <stdio.h>
int main(void)
{
int n = 50;
printf("%i\n", n);
}
Notice how
n
is stored in memory with the value of50
.
We can visualize how the program stores this value as follows:
0x123
being the hexadecimal address where the value is stored.
The C language has to powerful operators that relate to memory:
-
&
The address of operator, provides the address of something stored in memory. -
*
The dereference operator, instructs the compiler to go to a location in memory.
Let's apply this to our code:
#include <stdio.h>
int main(void)
{
int n = 50;
printf("%p\n", &n);
}
-
%p
is format code to print an address. -
&n
is equivalent tothe address of n
.
This code returns the address in the computer memory where n
is stored:
0x7ffd077b19dc
A pointer is a variable that stores the memory address of another variable as its value.
int n = 50;
int *p = &n;
p
is a pointer that contains the memory address of an integern
.
#include <stdio.h>
int main(void)
{
int n = 50;
int *p = &n;
printf("%p\n", p);
}
0x7ffd437f067c
This code has the same effect as the previous code, but includes a pointer.
Let's explore the use of the *
operator:
#include <stdio.h>
int main(void)
{
int n = 50;
int *p = &n;
printf("%i\n", *p);
}
printf
line request: Go to the address *p
and print the integer value %i
stored there:
50
Note
There is a difference between the first and second *p
.
int *p
is syntax for declaring a pointer (a variable that will store an address).
*p
on the printf line just means go to location p
.
Let's visualize how the code is stored:
-
The integer
50
stored in the variablen
can be found at the address0x123
. -
The pointer variable
p
storing the address of n =&n
=0x123
, can be found at another location on the memory and is usually stored as an8-byte value
.
No that we understand the concept of pointers, let's revisit strings and dive deeper into how they are stored.
string s = "HI!"
We can visualize this string s
stored as follows:
| H | I | ! | \0 |
s[0] s[1] s[2] s[3]
0x123 0x124 0x125 0x126
-
In
string s = "HI!"
,s
stores the address of the beginning of the string. -
s
stores0x123
. -
s
is a pointer to the address ofs[0]
which holds the value of the first element of the string,H
.
Now that we know that s
is a pointer to an address, let's print out it's value using %p
:
#include <cs50.h>
#include <stdio.h>
int main(void)
{
string s = "HI!";
printf("%p\n", s);
}
0x564fed03a004
Let's also printout the addresses of the characters in s
using the operator &
:
#include <cs50.h>
#include <stdio.h>
int main(void)
{
string s = "HI!";
printf("%p\n", s);
printf("%p\n", &s[0]);
printf("%p\n", &s[1]);
printf("%p\n", &s[2]);
printf("%p\n", &s[3]);
}
0x564fed03a004 = address of s
0x564fed03a004 = address of H
0x564fed03a005 = address of I
0x564fed03a006 = address of !
0x564fed03a007 = address of \0
Note
The addresses corresponding to s
and s[0]
are the same. Remember that s
points to the first character of the string (H).
Notice also the last digit of the addresses. It shows that the elements are next to each other on the memory, 1 byte
apart.
The string
data type is a cs50.h
invention to introduce strings in a simplified way.
string s = "HI!";
In raw C
code, it looks like this:
char *s = "HI!";
-
char *
replaces thestring
data type. -
char
is data type for a single character. -
*
is the dereference operator that tells the compiler to go to that address. -
*s
is the address for the first character of the string calleds
.
Now, we can get rid of the cs50.h
library and write the code in raw C
and have the same result:
#include <stdio.h>
int main(void)
{
char *s = "HI!";
printf("%s\n", s);
}
HI!
string
data type was created with a single line of code, using a typedef declaration:
typedef char *string
This teaches the compiler that
string
is an alias ofchar *
.
#include <stdio.h>
int main(void)
{
char *s = "HI!";
printf("%c", s[0]);
printf("%c", s[1]);
printf("%c\n", s[2]);
}
HI!
The program above prints out the string HI!
by printing out each char
back-to-back.
#include <stdio.h>
int main(void)
{
char *s = "HI!";
printf("%c", *s);
printf("%c", *(s + 1));
printf("%c\n", *(s + 2));
}
HI!
Note
Since *s
points to the 1st char of the string, we can determine that *(s + 1)
is the location of the 2nd char, and *(s + 2)
the 3rd.
A string of characters is simply an array of characters identified by its first byte.
To compare integers, we used the ==
equality operator:
#include <cs50.h>
#include <stdio.h>
int main(void)
{
// Get two integers
int i = get_int("i: ");
int j = get_int("j: ");
// Compare integers
if (i == j)
{
printf("Same\n");
}
else
{
printf("Different\n");
}
}
However, in the case of strings, we cannot compare two strings using the ==
operator:
#include <cs50.h>
#include <stdio.h>
int main(void)
{
// Get two strings
char *s = get_string("s: ");
char *t = get_string("t: ");
// Compare strings' addresses
if (s == t)
{
printf("Same\n");
}
else
{
printf("Different\n");
}
}
Important
Using the ==
operator to compare strings will attempt to compare the memory locations of the strings instead of the characters.
s | t |
---|---|
s[0] | s[0] |
0x123 | 0x456 |
-
Different strings are located in different memory addresses.
-
String
s
could be located in address0x123
, while stringt
might be located in address0x456
. -
Typing the HI! as input to both prompts in the code above will still result in an output of
Different
.
Using strcmp
function of the string.h
library, we can correct our code:
#include <cs50.h>
#include <stdio.h>
#include <string.h>
int main(void)
{
// Get two strings
char *s = get_string("s: ");
char *t = get_string("t: ");
// Compare strings
if (strcmp(s, t) == 0)
{
printf("Same\n");
}
else
{
printf("Different\n");
}
}
Notice that
strcmp
takes the strings as arguments and can return0
if the strings are the same.
We can see that these two strings are located in different addresses using the %p
placeholder in the print statement:
#include <cs50.h>
#include <stdio.h>
int main(void)
{
// Get two strings
char *s = get_string("s: ");
char *t = get_string("t: ");
// Print strings
printf("%p\n", s);
printf("%p\n", t);
}
0x56211767d6b0
0x56211767d6f0
Note
We do not need to use the &s
or &t
like in other data types, because we now know that strings are already pointers
and hold the address of the first character of the string.
Let's try and copy one string to another:
#include <cs50.h>
#include <ctype.h>
#include <stdio.h>
#include <string.h>
int main(void)
{
string s = get_string("s: ");
string t = s;
t[0] = toupper(t[0]);
printf("%s\n", s);
printf("%s\n", t);
}
The program above attempts to:
- Copy string
s
into stringt
. - Then Capitalize the first character of string
t
. - And print out both strings.
Theoretically, if we input hi!
into the prompt, the program should print out:
hi!
Hi!
But this will not work, because string t = s
copies the address of s
to t
.
Both strings now hold the same address and point to the same block in memory t[0] == s[0]
. The output will be the following:
Hi!
Hi!
Before we address this bug, we can add a security layer and make sure the string t
has at least one character before attempting to capitalize its first letter. This will prevent a segmentation fault
from happening and our program crashing.
#include <cs50.h>
#include <ctype.h>
#include <stdio.h>
#include <string.h>
int main(void)
{
// Get a string
string s = get_string("s: ");
// Copy string's address
string t = s;
// Capitalize first letter in string
if (strlen(t) > 0)
{
t[0] = toupper(t[0]);
}
// Print string twice
printf("s: %s\n", s);
printf("t: %s\n", t);
}
if (strlen(t) > 0)
This condition checks is the length of the stringt
(number of characters) is greater than0
.
To be able to make an authentic copy of the string, we will need to use two new building blocks:
malloc
allows us to allocate a block of a specific size of memory.free
allows us to tell the compiler to free up that block of memory we previously allocated.
#include <cs50.h>
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
{
// Get string s
char *s = get_string("s: ");
// Allocate memory for another string
char *t = malloc(strlen(s) + 1);
// Copy string into memory, including "\0"
for (int i = 0, n = strlen(s); i <= n; i++)
{
t[i] = s[i];
}
// Capitalize copy
if (strlen(t) > 0)
{
t[0] = toupper(t[0]);
}
// Print strings
printf("%s\n", s);
printf("%s\n", t);
}
Important
We have to ensure that we include the null \0
character in our copied string.
-
malloc(strlen(s) = 1)
creates a block of memory that is the length of the strings
plus 1. This ensures the inclusion of the null\0
character. -
The
for
loop iterates through strings
indexes and assigns each value to the same locations on stringt
. -
To prevent running a function over and over again, in the for loop, we did not call the
strlen(s)
function in the middle of the condition like soi <= strlen(n)
. -
Instead, we declared
n = strlen(s)
and used the conditioni <= n
. This ensuresstrlen
only runs once.
If something goes wrong and we are out of memory in the computer, both malloc
and get_string
functions return NULL
.
We can check for this condition and exit the program early, adding a layer of safety as follows:
#include <cs50.h>
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
{
char *s = get_string("s: ");
// Check for NULL
if (t == NULL)
{
return 1;
}
char *t = malloc(strlen(s) + 1);
// Check for NULL
if (t == NULL)
{
return 1;
}
for (int i = 0, n = strlen(s); i <= n; i++)
{
t[i] = s[i];
}
if (strlen(t) > 0)
{
t[0] = toupper(t[0]);
}
printf("%s\n", s);
printf("%s\n", t);
}
The C
language has a built-in function to copy strings called strcpy
. It can replace ouf for loop
as follows:
#include <cs50.h>
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
{
char *s = get_string("s: ");
if (t == NULL)
{
return 1;
}
char *t = malloc(strlen(s) + 1);
if (t == NULL)
{
return 1;
}
// Copy string into memory
strcpy(t, s);
if (strlen(t) > 0)
{
t[0] = toupper(t[0]);
}
printf("%s\n", s);
printf("%s\n", t);
}
In C, the free
function is used to deallocate memory that was previously allocated using the function malloc
.
This is crucial to prevent memory leaks and efficiently managing memory in C programs.
#include <cs50.h>
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
{
char *s = get_string("s: ");
if (t == NULL)
{
return 1;
}
char *t = malloc(strlen(s) + 1);
if (t == NULL)
{
return 1;
}
strcpy(t, s);
if (strlen(t) > 0)
{
t[0] = toupper(t[0]);
}
printf("%s\n", s);
printf("%s\n", t);
// Free memory allocated to `t`
free(t);
return 0;
}
Note
The program execution is complete once the printf
statements are executed. After that the program terminates, making t
no longer needed for future operations.
t
can now be safely deallocated using the free
function.
Valgrind is a tool that we can use to spot memory-related issues. Specifically if we effectively free
memory previously allocated in the program with malloc
.
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int *x = malloc(3 * sizeof(int));
x[1] = 72;
x[2] = 73;
x[3] = 33;
}
-
int *x
Declares the address of an integer. -
= malloc(3 * sizeof(int));
Assigns to it enough space for an array of size 3 integers. -
sizeof(int)
Instead of guessing how many bytes does an integer take in a specific machine, this function will take care of that for us. -
Then proceeds to assign values to the indexes of the array.
If we compile the code and type valgrind ./file_name
, we will get a report that will indicate the following errors:
==18341== Invalid write of size 4
==18341== at 0x109170: main (memory.c:9)
==18341== Address 0x4bb404c is 0 bytes after a block of size 12 alloc'd
==18341== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==18341== by 0x109151: main (memory.c:6)
- In
line 9
we attempted to assign the value of33
at the 4th position of the array, but we only allocated an array of size3
. We should have started to assign values at the first indexx[0]
.
==18341== 12 bytes in 1 blocks are definitely lost in loss record 1 of 1
==18341== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==18341== by 0x109151: main (memory.c:6)
- This error indicates a
memory leak
. Inline 6
we allocated12 bytes
of memory that were lost. We never usedfree
to deallocatex
.
We can fix the program by modifying the code as follows:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
int *x = malloc(3 * sizeof(int));
x[0] = 72;
x[1] = 73;
x[2] = 33;
free(x);
}
And if we run valgrind ./file_name
again:
==27591== HEAP SUMMARY:
==27591== in use at exit: 0 bytes in 0 blocks
==27591== total heap usage: 1 allocs, 1 frees, 12 bytes allocated
==27591==
==27591== All heap blocks were freed -- no leaks are possible
Garbage values, are values stored in a variable or memory location that has not been explicitly initialized or assigned with a meaningful value.
When you ask the compiler for a block of memory, there is no guarantee that this memory will be empty. It's very possible that this memory contains a garbage value.
#include <stdio.h>
int main(void)
{
int scores[1024];
for (int i = 0; i < 1024; i++)
{
printf("%i\n", scores[i]);
}
}
When we run this code, some of the values (not all 1024) returned below:
0
1685382482
4
2185456
0
2189552
0
2189552
0
14096
0
14096
-
In this program we allocated
1024
locations in memory for our array. -
The values in this array have not been initialized.
-
When running the program, we notice that it prints some garbage values and not just
0s
.
Let's see another problematic scenario:
int main(void)
{
int *x;
int *y;
x = malloc(sizeof(int));
*x = 42;
*y = 13;
y = x;
*y = 13
}
-
In this function, we declared
*x
and*y
pointers. -
Only allocated memory for
*x
. -
We dereference
*x
to store in it the value42
. -
But then, attempted to dereference
*y
to assign to it the value13
. -
Since we never initialized
*y
, it most likely contains agarbage value
. -
Attempting to assign
13
to*y
could lead to asegmentation fault
or another unexpected behavior because we don't know where*y
points. -
We then assign the value of x to y with
y = x
. This means that the random memory allocated fory
(if any) is lost, leading to a memory leak.
The simple solution here that will make this code correct is simply removing line *y = 13;
. This would make next line y = x;
valid and *x
and *y
point to the same location. Assigning value of 13
to *y
in the last line will change the value stored in the common memory location.
1. Pointer and pointee (memory location) are separate - don't forget to set up a pointee.
2. Pointer dereferencing (assign value) to access its pointee.
3. Pointer assignment (=) makes pointers point to the same memory location.
A common need in programming is to swap the values of two variables. An important step in this process is to temporarily hold space.
Let's see this process in action:
#include <stdio.h>
void swap(int a, int b);
int main(void)
{
int x = 1;
int y = 2;
printf("x is %i, y is %i\n", x, y);
swap(x, y);
printf("x is %i, y is %i\n", x, y);
}
void swap(int a, int b)
{
int tmp = a;
a = b;
b = tmp;
}
-
After printing the values of
x
andy
a first time. -
This program attempts to swap the values of
x
andy
using the swap function. -
swap()
- Assigns the value of
x
in a temporary variabletmp
to free up space inx
. - Assigns the value of
y
tox
. - Now that
y
is free, it assigns to it the old value ofx
stored intmp
. - Theoretically achieving to swap
x
andy
.
- Assigns the value of
But there is an issue here, when we run the program it fails to swap the values:
x is 1, y is 2
x is 1, y is 2
This is an issue of scope
. Remember that local variables are passed by value
and changes made to them in another function swap()
, do not affect the original variables in the calling function main()
.
When we used the swap()
function we swapped "copies" of x
and y
.
To illustrate this, let's breakdown how information in stored in the computer's memory:
-
Machine code
At the lowest level we have machine code or instructions that have been compiled and represent the operations to be performed by the CPU. -
Global variables
Variables that are initialized before the program starts, outside of themain()
function and every other function. These variables are accessible from anywhere in the program and stored in global memory. -
Heap
The heap is a region of memory used for dynamic memory allocation. Memory allocated on the heap usingmalloc
persists until it is explicitly deallocated withfree
. -
Stack
is the region of memory used for function call management and local variable storage. Each time a function is created, a newstack frame
is created containing all the information related to the function call (parameters, local variables, return address, etc).
Note
The main()
function and the swap()
function have two different stack frames in the memory. When passing variables as arguments to a function in C
, that function receives copies. Therefore, modifying theses copies inside the function does not affect the original variables in the caller's context.
To enable swap()
to modify the original variables passed to it, we need to pass by reference
using pointers. This way, the function can dereference the pointers and access the actual memory location of the variables.
void swap(int *a, int *b)
{
int tmp = *a;
*a = *b;
*b = tmp;
}
-
By adding the
*
operator to the arguments, we make them pointers andswap()
has access to the addresses ofx
andy
and not just a copy of their values. -
*a
points tox
and*b
points toy
.
We also need to add the &
operator to x
and y
in the function call, to ensure that we pass their addresses as arguments.
swap(&x, &y);
Now that swap()
can access the actual memory location of x
and y
, the changes it makes are reflected in the main()
function and the output of the program is the following:
x is 1, y is 2
x is 2, y is 1
Heap overflow
occurs when a program writes data beyond the memory allocated to it on the heap, touching areas of memory it is not supposed to. Typically happens when we dynamically allocate malloc()
memory but write more data than the allocated.
Stack overflow
is when too many functions are called, overflowing the amount of memory available on the stack.
Each time a function is called a new stack frame is added to the call stack. If a program recursively calls a function without proper termination condition, it can lead to stack overflow.
In CS50 functions like get_int
and get_string
have been created to simplify the act of getting user input. In C we can use the scanf
built-in function to get user input.
Let's implement the function using raw C code:
#include <stdio.h>
int main(void)
{
int n;
printf("n: ");
scanf("%i", &n);
printf("n: %i\n", n);
}
scanf("%i", &n)
stores the user input value ofn
at the location&n
.
Let's try to implement this function using raw C code:
#include <stdio.h>
int main(void)
{
char *s;
printf("s: ");
scanf("%s", s);
printf("s: %s\n", s);
}
Notice that no
&
is required because strings are already represented using pointers to the first character of the string.
If we succeed in compiling this program, we will get an error:
Segmentation fault (core dumped)
- We declared the pointer variable
s
without initializing it (allocating memory to it). This means that it does not point to any valid memory address. Instead, it contains agarbage value
(it points to some random memory location which may not be accessible or safe to use).
To fix this, we can modify the code as follows:
#include <stdio.h>
int main(void)
{
char s[4];
printf("s: ");
scanf("%s", s);
printf("s: %s\n", s);
}
-
However, we had to pre-allocate a certain amount of memory, in this case
s[4]
. This is only a guess, because we cannot know what size string the user is going to input. -
Declaring the string as
s[4]
, we allocated enough memory to hold four characters of1 byte
each (s[0], s[1], s[2]) including the null terminator\0
(s[3]). Totaling4 bytes
.
If the user provides a longer string as an input, we will step out of the bounds of the memory allocated resulting in an error:
Segmentation fault (core dumped)
It is risky to use the scanf
function in general and especially with strings, because it lacks the memory error handling and can lead to buffer overflows. CS50's get_string
function uses malloc
recursively to resize and allocate more data depending on the user input.
File Input/Output refers to the operations of reading from and writing to files on a computer's storage system.
#include <cs50.h>
#include <stdio.h>
#include <string.h>
int main(void)
{
// Open csv file
FILE *file = fopen("phonebook.csv", "a");
// Get name and number
char *name = get_string("Name: ");
char *number = get_string("Number: ");
// Print to file
fprintf(file, "%s,%s\n", name, number);
// Close file
fclose(file);
}
Notice that the program uses pointers to access the file.
-
We can create a file called
phonebook.csv
before running this program. After running, the program will prompt user for aname
andnumber
that it will write in the CSV file. -
FILE *file
is a pointer to the file in memory. -
fopen()
function opens the file. -
With
"a"
, we specify that we want to append the file. -
fprintf()
prints to the file two strings (name and number). -
fclose()
closes the file.
If we want to ensure that phonebook.csv
exists prior to running the program, we can modify our code as follows:
#include <cs50.h>
#include <stdio.h>
#include <string.h>
int main(void)
{
// Open csv file
FILE *file = fopen("phonebook.csv", "a");
// Check if file exists
// Alternative -> if (file == NULL)
if (!file)
{
return 1;
}
// Get name and number
char *name = get_string("Name: ");
char *number = get_string("Number: ");
// Print to file
fprintf(file, "%s,%s\n", name, number);
// Close file
fclose(file);
}
- The
if (!file)
condition protects the program against aNULL
pointer by invokingreturn 1
.
Let's implement our own copy program for files:
#include <stdio.h>
#include <stdint.h>
typedef uint8_t BYTE;
int main(int argc, char *argv[])
{
FILE *src = fopen(argv[1], "rb");
FILE *dst = fopen(argv[2], "wb");
BYTE b;
while (fread(&b, sizeof(b), 1, src) !=0)
{
fwrite(&b, sizeof(b), 1, dst);
}
fclose(dst);
fclose(src);
}
-
typedef uint8_t BYTE
means give me an "unsigned" (no negative number), 8 bit value inside a new type called BYTE. -
FILE *src = fopen(argv[1], "rb");
Declare a pointer variable called src of type FILE, and open the file specified in the 2nd command line argument to read binary. -
FILE *dst = fopen(argv[2], "wb");
Declare a second pointer variable called dst of type FILE, and open the file specified in the 3rd command line argument to write binary. -
BYTE b
store 1 byte declared earlier in a variable called b. -
while(...) !=0
While fread() is successful continue. -
fread(&b, sizeof(b), 1, src
Read bites and store them in address of b, of size 1 bite, one byte at a time from the src file. -
fwrite(&b, sizeof(b), 1, dst);
Write bites stored in address of b, of size 1 byte, one bite at a time, to dst file. -
fclose
close files dst and src.
With this simple but powerful copy program implemented in C we can now copy the contents of any binary file to another file byte per byte.
In the problem sets we will be manipulating Bitmap Image Files (BMPs).