forked from LiamLuck/ProgrammingPractical
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathTom.html
229 lines (225 loc) · 17.9 KB
/
Tom.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" href="favicon.png" type="image/x-icon">
<title>Media Manipulation</title>
<link href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-1BmE4kWBq78iYhFldvKuhfTAU6auU8tT94WrHftjDbrCEXSU1oBoqyl2QvZ6jIW3" crossorigin="anonymous">
<link href="style.css" rel="stylesheet">
</head>
<body>
<div class="topnav">
<a href="index.html">Home Page</a>
<a href="liam.html">Virtual Reality</a>
<a href="sam.html">Artificial Intelligence</a>
<a class="active" href="Tom.html">Media Manipulation</a>
<a href="Harrison.html">Educational Uses</a>
</div>
<h1>Uses of Media Manipulation</h1>
<p style="width:95%">Broken down to its simplest form, an image is just a collection of pixels with a certain resolution and file format. A video is just a collection of images strung together with certain parameters. With these forms of media being as simple as they are, it's relatively simple to manipulate them using basic computer-enabled tools and algorithms, for a vast array of purposes and results.</p>
<h2>Basic adjustment</h2>
<h3>Images</h3>
<div class="row">
<div class="basiccolumn" style="width: 50%; margin-right: 2%;">
<p>Given that the pixels comprising an image are defined by only a few numerical values, it's easy for a computer to adjust those values to create a certain effect.</p>
<p>Some of the simplest changes that can be made are:</p>
<ul>
<li>Brightness</li>
<li>Contrast</li>
<li>Hue</li>
</ul>
<p>And all of these can be achieved by shifting the three RGB colour values.</p>
<p>For example, with 8-bit RGB colour space, a pixel may be defined as (x,y,z), where all three values are in the interval [0,255].</p>
<p>Increasing brightness would involve increasing all 3 values, increasing contrast would involve moving the values further from the median point, and changing hue would involve increasing some values and decreasing others.</p>
<p>These basic features and others like them are present on most consumer-grade image handling software, such as Photoshop, Instagram, and even Preview.</p>
</div>
<div class="basiccolumn" style="width: 45%;">
<figure style="margin: 0%; margin-top: 2%;">
<img src="images/histogram.png" alt="An image with histogram before and after the contrast is raised" width="630px">
<figcaption><strong>1.1</strong> An image given increased contrast, shown by the range of histogram values</figcaption>
</figure>
</div>
</div>
<div class="row">
<div class="basiccolumn" style="width: 50%; margin-right: 5%;">
<p><br>More complex algorithms are able to produce more interesting effects, such as:</p>
<ul>
<li>
Removing the background of an image by setting all pixels in a certain area to be completely transparent
<ul>
<li>Transparency is controlled by a forth value, called the alpha channel</li>
<li>Identifying the area to be altered is usually the most difficult part, and often requires manual adjustment by the user</li>
</ul>
</li>
<li>
Converting an image from one filetype to another
<ul>
<li>Different image filetypes support different attributes, such as transparency (present in .png files but not .jpg files)</li>
<li>Different filetypes often use different levels of image compression, which reduces the file size and sometimes loses visual information in the process</li>
</ul>
</li>
<li>
Identifying faces or other recognisable objects in an image
<ul>
<li>
This is exceptionally useful for algorithms which automatically organise and label collections of images, such as Apple's Photos app
</li>
<li>
However, this technology still has a long way to go, as demonstrated by the relative effectiveness of CAPTCHA tests and the common mistakes made by content filters (such as believing sand dunes to be inappropriate images)
</li>
</ul>
</li>
<li>
Noise reduction
<ul>
<li>Able to reverse the effects of compression, such as JPEG artifacts</li>
<li>Also able to clean up the effects of electronic noise, baked into the image by the camera that captured it</li>
<li>Excessive noise reduction can lead to a loss of detail</li>
</ul>
</li>
</ul>
<p>All these tools assist image analysis and make it far easier to produce better-looking images in a format suitable for their intended use. Advancements made in basic image-adjustment algorithms are incredibly useful and are frequently used by the vast majority of people in digitally developed nations.<br><br></p>
</div>
<div class="basiccolumn" style="width: 40%;">
<figure style="margin: 0%; margin-top: 20%;">
<img src="images/noise.jpg" alt="A visually noisy image" width="500px">
<figcaption><strong>1.2</strong> Clearly visable noise in a digitally captured image</figcaption>
</figure>
</div>
</div>
<h3>Videos</h3>
<div class="row">
<div class="basiccolumn" style="width: 30%;">
<p>Algorithm-based video editing is much less widely used, but is just as useful for the people who need it.</p>
<p>Many consumer-grade algorithms of this nature are included in high-end editing software, such as Premiere, After Effects, and Final Cut Pro.</p>
<p>Some features enabled by algorithms are:</p>
<ul>
<li>Video stabilisation</li>
<li>Motion blur</li>
<li>Increasing framerate</li>
</ul>
</div>
<div class="basiccolumn" style="width: 36%;">
<figure style="margin: 0%; margin-top: 3%;">
<img src="images/motionblur.jpg" alt="Motion blur added to a video in post" width="510px">
<figcaption><strong>2.2</strong> A video given motion blur by Adobe After Effects</figcaption>
</figure>
</div>
<div class="basiccolumn" style="width: 30%;">
<figure style="margin: 0%; margin-top: 3%;">
<img src="images/stabilisation.jpg" alt="Editing software carrying out video stabilisation" width="370px">
<figcaption><strong>2.1</strong> A stabilisation algorithm which keeps the recognisable pixels in the red boxes inside the green boxes, by subtly stretching and moving each frame</figcaption>
</figure>
</div>
</div>
<div class="row">
<div class="basiccolumn" style="width: 53%; margin-right: 5%;">
<p>There have been concerns raised over the use of modern editing capabilities to 'improve' older video files, such as in the following instances:</p>
<ul>
<li>
George Lucas's 'Special Editions' of the original Star Wars Trilogy and 'Director's Cut' of his previous film THX-1138
<ul>
<li>These made use of modern visual effects and compositing technology to alter many scenes, and to insert some entirely new ones.</li>
<li>The changes made are widely criticised, mostly due to the fact that the original versions of the films are no longer widely available</li>
<li>The fact that modern computing technology can be used to essentially overwrite previously existing versions of films is therefore a great source of concern for many</li>
</ul>
</li>
<li>
The artificial increase of the framerate of videos through AI-based interpolation
<ul>
<li>This is made possible through software such as <a href="https://grisk.itch.io/dain-app">DAIN</a>, which analyses the movement of pixels from frame to frame to interpret images that may have come in-between</li>
<li>However, many believe this infringes on the artistic integrity of hand-animated videos</li>
<li>The software both overwrites the original sequence of frames, which were drawn by the artist to portray motion of a certain energy and style, and occasionally inserts frames with noticable 'smears' and inconsistencies</li>
</ul>
</li>
</ul>
</div>
<div class="basiccolumn" style="width: 35%;">
<figure style="margin-left: 0%; margin-top: 10%;">
<img src="images/specialedition.jpg" alt="Changes made in the special edition" width="500px">
<figcaption><strong>2.3</strong> Top: The original ending of Star Wars: Return of the Jedi / Bottom: The 'Special Edition'</figcaption>
</figure>
</div>
</div>
<div class="row" style="margin-top: 2%;">
<div class="basiccolumn" style="width: 35%; margin-right: 5%;">
<figure style="margin-left: 3%;">
<img src="images/cat.gif" alt="Frame interpolation used on an animation">
<figcaption><strong>2.5</strong> The original video is on the left, and the other video was created through frame interpolation</figcaption>
</figure>
</div>
<div class="basiccolumn" style="width: 50%;">
<figure style="margin-left: 0%; margin-top: 5%;">
<img src="images/interpolation.gif" alt="An example of frame interpolation" style="width: 700px;">
<figcaption><strong>2.4</strong> The original video is on the left, and the others are created through frame interpolation</figcaption>
</figure>
</div>
</div>
<h2 style="margin-top: 3%;">More advanced applications</h2>
<h3>Digital art</h3>
<p style="width: 95%;">All the applications of computer algorithms discussed so far have aimed to enhance media and media analysis without changing the basic purpose of the media. While framerate interpolation and 'special editions' of films may slightly alter the artistic intent of the work, the work is still intended to be viewed in the same fashion by its consumers. However, digital manipulation can also be used to alter a piece of media to such an extent that the manipulation becomes a work of artistic expression in its own right.</p>
<p style="width: 95%;">Consider, for example, the AI-enabled tools available at the <a href="https://deepdreamgenerator.com/">Deep Dream Generator</a>, which are able to blend together two images/textures in a variety of ways to produce a completely visually distinct result.</p>
<p style="width: 95%; margin-bottom: 2%;">These tools emerged from the original 'Deep Dream' tool, which was initially "invented to help scientists and engineers to see what a deep neural network is seeing when it is looking in a given image", but evolved into "a new form of psychedelic and abstract art".</p>
<div class="row">
<div class="basiccolumn" style="width: 28%;">
<figure style="margin-left: 0%;">
<img src="images/deepstyle.jpg" alt="A merging of a woman's face and a painting of the ocean" width="390px">
<figcaption><strong>3.1</strong> An application of the 'Deep Style' tool</figcaption>
</figure>
</div>
<div class="basiccolumn" style="width: 28%;">
<figure style="margin-left: 0%;">
<img src="images/thinstyle.jpg" alt="A merging of a man's face and a painting of a cottage" width="390px">
<figcaption><strong>3.2</strong> An application of the 'Thin Style' tool</figcaption>
</figure>
</div>
<div class="basiccolumn" style="width: 30%;">
<figure style="margin-left: 0%;">
<img src="images/deepdream.jpg" alt="A merging of architecture, eyeballs, and animals" width="490px">
<figcaption><strong>3.3</strong> An application of the original 'Deep Dream' tool</figcaption>
</figure>
</div>
</div>
<div class="row" style="margin-top: 1%;">
<div class="basiccolumn" style="width: 35%; margin-right: 2%;">
<p>A simpler use of modern image manipulation software is animating still images. The Deep Nostalgia tool provided by MyHeritage is perhaps the most well-known example.</p>
<p>This technology was developed for the purpose of animating old family photos, as displayed on the <a href="https://www.myheritage.com/deep-nostalgia">MyHeritage homepage</a>, but works on any image containing a recognisable face. While the technology on display here is certainly impressive, there remains an eerie quality to the final result, similar to the 'uncanny valley' effect of replications of the human form which are <strong>near</strong>-identical.</p>
<p>Displayed in this section are just a handful of the many artistic applications of image manipulation algorithms. Advancements in computer science have enabled the emergence of new forms of art, evolved from the building blocks of simple image adjustment techniques.<br><br></p>
</div>
<div class="basiccolumn" style="width: 29%;">
<figure style="margin-left: 0%; margin-top: 2%;">
<img src="images/skywalkerstill.jpg" alt="A picture of Luke Skywalker" width="390px">
<figcaption><strong>3.4</strong> The original image</figcaption>
</figure>
</div>
<div class="basiccolumn" style="width: 30%;">
<figure style="margin-top: 2%;">
<img src="images/skywalkermove.gif" alt="An animated video of Luke Skywalker" width="390px">
<figcaption><strong>3.5</strong> The image given motion by MyHeritage</figcaption>
</figure>
</div>
</div>
<h3>Deepfakes and ethics</h3>
<p style="width: 95%;">Perhaps the most impressive use of modern computing technology to manipulate media is also its most controversial use. An AI is fed a large number of images of a person's face, usually taken from individual frames of a video, to give it an understanding of what they look like from multiple angles and under different lighting conditions.</p>
<p style="width: 95%;">Using this data of person A, along with data of changing facial expressions and alignment taken from a video of some person B, the video of person B can be adjusted to give them the face of person A.</p>
<div class="row">
<div class="basiccolumn" style="width: 38%;">
<figure style="margin-left: 0%;">
<img src="images/frames.jpg" alt="Frames of Matt Parker's face" width="514px">
<figcaption><strong>4.1</strong> A collection of frames taken from a video of Matt Parker</figcaption>
</figure>
</div>
<div class="basiccolumn" style="width: 45%;">
<figure style="margin-left: 0%;">
<img src="images/amycage.gif" alt="Amy Adams with Nicolas Cage's face" width="450px">
<figcaption><strong>4.2</strong>Nicolas Cage's face deepfaked onto Amy Adams</figcaption>
</figure>
</div>
</div>
<p style="width: 95%;">While the above example is relatively harmless, the ethics of using a person's likeness to insert them into a scene without their consent is highly questionable, especially when this technology is used to create pornography, as roughly 96% of online deepfakes are.</p>
<p style="width: 95%; margin-bottom: 2%;">Thankfully, processing power is still a barrier to most amateur deepfake production. The analysis and re-rendering of thousands of frames of video prevents most low-end hardware from realistically being able to produce deepfakes, although modern hardware is advancing fast enough that things may not stay that way for long.</p>
<p style="margin-top: 1%;">Media manipulation technology has come a long way from early days of image adjustment, for the better and the worse. Putting aside the ethical implications of some of the most advanced uses of computer science in this field, strides made have opened the doors to far better looking images and videos, as well whole new genres of digital art and truly impressive synthetic media. Where the technology will go from here is impossible to predict, but this is a field of computing that will only continue to evolve.</p>
<figcaption style="margin-top: 3%">Authored by Tom Lister: <a href="https://github.com/Tom-lister">Github</a></figcaption>
</body>
</html>