README.md 10.2 KB
Newer Older
Chipp Jansen's avatar
Chipp Jansen committed
1
2
3
4
# Conversations with Cameras

AMRO 2022 Workshop Repository for Documentation

Chipp Jansen's avatar
docs    
Chipp Jansen committed
5
## Connect to your camera
Chipp Jansen's avatar
Chipp Jansen committed
6

Chipp Jansen's avatar
docs    
Chipp Jansen committed
7
We will use `ssh` (Secure Shell) to connect to the camera.
Chipp Jansen's avatar
Chipp Jansen committed
8

Chipp Jansen's avatar
docs    
Chipp Jansen committed
9
10
You will need to find a terminal program (on a Linux or Mac)
on your computer where you can run `ssh` from.  On a Windows laptop you can use putty to connect with `ssh`.  Let me know if you need help with getting started with `ssh`, we can pair you up with someone who can help.
Chipp Jansen's avatar
Chipp Jansen committed
11

Chipp Jansen's avatar
docs    
Chipp Jansen committed
12
With the camera's IP address use this command to connect to the camera (this example uses 192.168.1.20 as the IP address, replace it with your camera's IP):
Chipp Jansen's avatar
Chipp Jansen committed
13

Chipp Jansen's avatar
Chipp Jansen committed
14
    ssh root@192.168.1.20
Chipp Jansen's avatar
Chipp Jansen committed
15

Chipp Jansen's avatar
docs    
Chipp Jansen committed
16
The first thime you do tihs you might get something that says this:
Chipp Jansen's avatar
Chipp Jansen committed
17

Chipp Jansen's avatar
Chipp Jansen committed
18
19
20
    The authenticity of host '192.168.1.20 (192.168.1.20)' can't be established.
    ECDSA key fingerprint is SHA256:8WvDPcIb1JWa7EY34QHHwRV48vnLakWiJ1xTnz9+hJs.
    Are you sure you want to continue connecting (yes/no/[fingerprint])?
Chipp Jansen's avatar
Chipp Jansen committed
21

Chipp Jansen's avatar
docs    
Chipp Jansen committed
22
Type `yes` and hit ENTER.
Chipp Jansen's avatar
Chipp Jansen committed
23

Chipp Jansen's avatar
docs    
Chipp Jansen committed
24
The password to use is `root`.
Chipp Jansen's avatar
Chipp Jansen committed
25

Chipp Jansen's avatar
docs    
Chipp Jansen committed
26
If successful, you will get a command prompt that looks like this:
Chipp Jansen's avatar
Chipp Jansen committed
27

Chipp Jansen's avatar
Chipp Jansen committed
28
    #
Chipp Jansen's avatar
Chipp Jansen committed
29

Chipp Jansen's avatar
docs    
Chipp Jansen committed
30
You can check which camera you are on by reading the hostname file on the camera:
Chipp Jansen's avatar
Chipp Jansen committed
31

Chipp Jansen's avatar
Chipp Jansen committed
32
    more /etc/hostname
Chipp Jansen's avatar
Chipp Jansen committed
33

Chipp Jansen's avatar
docs    
Chipp Jansen committed
34
It should be the name that is on the camera.  If it does not match, you might have connected to someone elses camera!  You can type the command `exit` to close the connection and try the `ssh` command again with a different IP address.
Chipp Jansen's avatar
Chipp Jansen committed
35

Chipp Jansen's avatar
docs    
Chipp Jansen committed
36
## Set-up of the camera
Chipp Jansen's avatar
Chipp Jansen committed
37

Chipp Jansen's avatar
docs    
Chipp Jansen committed
38
The camera filesystem is set-up with a "traditional" Linux filesystem.  You can list the directories at the root of the file system:
Chipp Jansen's avatar
Chipp Jansen committed
39

Chipp Jansen's avatar
Chipp Jansen committed
40
    ls /
Chipp Jansen's avatar
Chipp Jansen committed
41

Chipp Jansen's avatar
docs    
Chipp Jansen committed
42
There is a directory `/sdcard` which is the sdcard of the camera.  If you don't see a sdcard directory, let me know.  You will need it to run the programs on the camera.
Chipp Jansen's avatar
Chipp Jansen committed
43

Chipp Jansen's avatar
docs    
Chipp Jansen committed
44
Change directory to the sdcard to see what is there.
Chipp Jansen's avatar
Chipp Jansen committed
45

Chipp Jansen's avatar
Chipp Jansen committed
46
47
    cd /sdcard
    ls
Chipp Jansen's avatar
Chipp Jansen committed
48

Chipp Jansen's avatar
docs    
Chipp Jansen committed
49
## See what is running on the camera
Chipp Jansen's avatar
Chipp Jansen committed
50

Chipp Jansen's avatar
docs    
Chipp Jansen committed
51
You can see what programs are running on the camera and other activity statistics by running `htop`.  
Chipp Jansen's avatar
Chipp Jansen committed
52

Chipp Jansen's avatar
Chipp Jansen committed
53
    htop
Chipp Jansen's avatar
Chipp Jansen committed
54

Chipp Jansen's avatar
docs    
Chipp Jansen committed
55
You will see a listing like this, which updates periodically to show what is running.
Chipp Jansen's avatar
Chipp Jansen committed
56

Chipp Jansen's avatar
docs    
Chipp Jansen committed
57
You can use the up/down arrows to see the different programs running.  Notice the `videocapture`, `mjpg_streamer` and `rtsp...`.
Chipp Jansen's avatar
Chipp Jansen committed
58

Chipp Jansen's avatar
docs    
Chipp Jansen committed
59
## View the stream
Chipp Jansen's avatar
Chipp Jansen committed
60

Chipp Jansen's avatar
docs    
Chipp Jansen committed
61
If you have `mjpg_streamer` running, then you can view the stream, by openning a browser to (again, substitute your IP instead of 192.168.1.20):
Chipp Jansen's avatar
Chipp Jansen committed
62

Chipp Jansen's avatar
Chipp Jansen committed
63
    http://192.168.1.20:8080/?action=stream
Chipp Jansen's avatar
Chipp Jansen committed
64

Chipp Jansen's avatar
docs    
Chipp Jansen committed
65
## Grab snapshots of the stream
Chipp Jansen's avatar
Chipp Jansen committed
66

Chipp Jansen's avatar
docs    
Chipp Jansen committed
67
`mjpg_streamer` also supports grabbing single JPGs of the stream.  You can grab an image using the `curl` utility, which let's you do HTTP requests (i.e. grab web-pages) from the command line.
Chipp Jansen's avatar
Chipp Jansen committed
68

Chipp Jansen's avatar
new    
Chipp Jansen committed
69
    curl -o /sdcard/snaps/image.jpg http://localhost:8080/?action=snapshot
Chipp Jansen's avatar
docs    
Chipp Jansen committed
70
71
72

This downloads a jpg into the `snaps` directory on the sdcard.

Chipp Jansen's avatar
Chipp Jansen committed
73
    ls /sdcard/snaps
Chipp Jansen's avatar
docs    
Chipp Jansen committed
74
75
76
77
78
79
80

Note that instead of using your specific IP, we can use `localhost` in the URL to substitute your own IP address when running programs from your camera.

## Stream that JPG!

Now we will set-up our own seperate hacky stream that will display this jpg.  This will run on a seperate port (note the 8081 in the command) on the camera:

Chipp Jansen's avatar
Chipp Jansen committed
81
    mjpg_streamer -i "input_file.so -f /sdcard/snaps" -o "output_http.so -p 8081" > /dev/null 2>&1 &
Chipp Jansen's avatar
docs    
Chipp Jansen committed
82
83
84
85
86
87
88

The `&` means this program will run in the background, and you can continue typing on the command prompt.  You can run `htop` and find this version of `mjpg_streamer` running.  We are running two seperate `mjpg_streamer` programs.

## Refresh that jpg

`mjpg_streamer` will watch the `snaps` directory for refreshed `test.jpg` to steam.  So we will use the `watch` command to re-run the `curl` command once a second (by setting the `-n 1` option).

Chipp Jansen's avatar
new    
Chipp Jansen committed
89
    watch -n 1 curl -o /sdcard/snaps/image.jpg http://localhost:8080/?action=snapshot
Chipp Jansen's avatar
docs    
Chipp Jansen committed
90

Chipp Jansen's avatar
Chipp Jansen committed
91
## View your hacky stream
Chipp Jansen's avatar
docs    
Chipp Jansen committed
92
93
94

Now using a slightly different url in the browser, we can view our every second "stream" (note that the only difference with this url is the port 8081 instead):

Chipp Jansen's avatar
Chipp Jansen committed
95
    http://192.168.1.20:8081/?action=stream
Chipp Jansen's avatar
docs    
Chipp Jansen committed
96

Chipp Jansen's avatar
Chipp Jansen committed
97
***Congratulations!*** You've set-up your own web stream.
Chipp Jansen's avatar
docs    
Chipp Jansen committed
98
99
100
101
102
103
104
105
106
107
108

## Before we modify the stream...

Now, we can manipulate the image from the camera every time we download it and the modified image is then streamed on a seperate.

But wait!  Unfortunately our little camera cannot handle manipulating the images at the current resolution, which is 1920 by 1080.  The camera will run out of memory and the conversion will crash!

So we need to get the `videocapture` program to streamer a smaller resolution for performance reasons.

First, we need to stop the current running programs.  You can easily stop ALL programs running with that name using `killall`.  So to stop `videocapture` and `mjpg_streamer`:

Chipp Jansen's avatar
Chipp Jansen committed
109
110
    killall videocapture
    killall mjpg_streamer
Chipp Jansen's avatar
docs    
Chipp Jansen committed
111
112
113
114
115

You can check `htop` to make sure they are stopped.

Next, we will restart `videocapture` with a configuration file (that is already on the sdcard at `/sdcard/mjpeg.json`) that sets it to stream at 320 by 180:

Chipp Jansen's avatar
Chipp Jansen committed
116
    /usr/bin/videocapture /sdcard/mjpeg.json >> /var/log/videocapture.log 2>&1 &
Chipp Jansen's avatar
docs    
Chipp Jansen committed
117
118
119

Next, we will restart streaming this original image with `mjpg_streamer`:

Chipp Jansen's avatar
Chipp Jansen committed
120
    mjpg_streamer -i "input_v4l2loopback.so -d /dev/video4 --fps 15" -o "output_http.so" > /dev/null 2>&1 &
Chipp Jansen's avatar
docs    
Chipp Jansen committed
121

Chipp Jansen's avatar
new    
Chipp Jansen committed
122
This is the exactly same command as before, and streams to port 8080.  We can refresh the browser to view this smaller stream (again, change to your IP address):
Chipp Jansen's avatar
docs    
Chipp Jansen committed
123

Chipp Jansen's avatar
Chipp Jansen committed
124
    http://192.168.1.20:8080/?action=stream
Chipp Jansen's avatar
docs    
Chipp Jansen committed
125
126
127

## Modify the stream

Chipp Jansen's avatar
new    
Chipp Jansen committed
128
Now we can modify `image.jpg` to produce a new image to stream.  A very handy tool to do this is [Imagemagick](https://imagemagick.org/), which is a power suite of command-line image editing tools.
Chipp Jansen's avatar
docs    
Chipp Jansen committed
129

Chipp Jansen's avatar
new    
Chipp Jansen committed
130
Now, we can use imagemagick's [`convert` program](https://imagemagick.org/script/magick.php) to manipulate and produce a new image that we can stream.  (Note that on the web documentation the `convert` program has been re-named to just `magick`.  We have an older version installed on the cameras so we will use `convert`)
Chipp Jansen's avatar
docs    
Chipp Jansen committed
131

Chipp Jansen's avatar
new    
Chipp Jansen committed
132
Let's test this out with a single image first.
Chipp Jansen's avatar
docs    
Chipp Jansen committed
133

Chipp Jansen's avatar
new    
Chipp Jansen committed
134
**Step 1** We grab this smaller image using curl (without `watch`) into the `snaps` directory (as before):
Chipp Jansen's avatar
docs    
Chipp Jansen committed
135

Chipp Jansen's avatar
new    
Chipp Jansen committed
136
    curl -o /sdcard/snaps/image.jpg http://localhost:8080/?action=snapshot
Chipp Jansen's avatar
docs    
Chipp Jansen committed
137

Chipp Jansen's avatar
new    
Chipp Jansen committed
138
**Step 2** We convert this image.  We will use the [`swirl` option](https://imagemagick.org/script/command-line-options.php#swirl)  which swirls the center of the image by a number of degrees. Imagemagick's `convert` has [SO MANY options](https://imagemagick.org/script/command-line-options.php) to manipulate the image.  For example, you can [draw](https://imagemagick.org/Usage/draw/) on the images.
Chipp Jansen's avatar
docs    
Chipp Jansen committed
139

Chipp Jansen's avatar
new    
Chipp Jansen committed
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
    convert /sdcard/snaps/image.jpg -swirl 60 /sdcard/stream/out.jpg

**Step 3** We will start a new `mjpg_streamer` which will stream images from the `stream` directory (Note that before we streamed from the original images directory).

    mjpg_streamer -i "input_file.so -f /sdcard/stream" -o "output_http.so -p 8081" > /dev/null 2>&1 &

Now, let's view this stream (on port 8081):

    http://192.168.1.20:8081/?action=stream

You might not see anyting at first.  With the browser open to this url, run the commands from  **Step 1** and **Step 2** again.  The image should show up now.

## Put it all together

Now that we need to run two commands, we will write a small script in `/sdcard` that we will run every second with `watch`.  The camera has `nano` which is a simple text editor (there is also `vi` for those who prefer that).  In the directory `/sdcard` run:

    nano swirl.sh

Now you can copy this script:

    #!/bin/sh
    curl -so /sdcard/snaps/image.jpg http://localhost:8080/?action=snapshot
    convert /sdcard/snaps/image.jpg -swirl 60 /sdcard/stream/out.jpg

You save by hitting `control-O` (the letter o), hitting Enter to accept.  Then `control-X` (the letter x) to exit.

Next, we need to make the script runnable as an executable (i.e. a program):

    chmod +x swirl.sh

Now we can run:

    watch -n 1 ./swirl.sh

Which runs the script every second.  But note that there is a delay on the camera to `curl` and `convert` the image!  

## Adding Randomness

Right now the swirl is the same.  Randomness can add some interest to the converted image.  We can use the `RANDOM` variable in our script to change the degree of the swirl.

First, we will start with a copy (`rswirl.sh`) of our script:

    cp swirl.sh rswirl.sh

With nano (`nano rswirl.sh`) we can edit the swirl degree with `RANDOM%360`, which is the random number mod 360.  (Mod is division but you keep the remainder only.  It's an easy way to create random numbers in a certain range of values 0 to 360).

Here is the `rswirl.sh` script:

    #!/bin/sh
    curl -so /sdcard/snaps/image.jpg http://localhost:8080/?action=snapshot
    convert /sdcard/snaps/image.jpg -swirl $((RANDOM%360)) /sdcard/stream/out.jpg

## Play with the options

There are so many options with `convert`.  Try picking one and modifying the image somehow.  For example you can tint the image a randome intensity of blue.

Here's a tip.  Create a new script with every modification that you are experimenting with.  
For example, create a new script (copy the old one) called someting like `rtint.sh`:

    #!/bin/sh
    curl -so /sdcard/snaps/image.jpg http://localhost:8080/?action=snapshot
    convert /sdcard/snaps/image.jpg -fill blue -tint $((RANDOM%100)) /sdcard/stream/out.jpg

## Further Ideas and Challenges
Chipp Jansen's avatar
fin    
Chipp Jansen committed
204
***Congratulations!!!*** You have successfully set-up your own modified camera image stream.  Although, it is slow, using `convert` is a flexible way of trying out modifying video streams.  The flexibility also means that it is slow, as `convert` runs as a general purpose program.  Often cameras such as these have specialised hardware that speeds up the performance greatly.
Chipp Jansen's avatar
new    
Chipp Jansen committed
205

Chipp Jansen's avatar
fin    
Chipp Jansen committed
206
207
208
  * **Try to draw something**  The [`draw` option](https://imagemagick.org/script/command-line-options.php?#draw) is very extensive!  It is easier to copy an example.  Try something simple like a drawing a circle.
  * **Composite** You can overlap one image over another using the [`composite`](https://imagemagick.org/script/composite.php) program.  You can use `curl` to grab an image from the internet.  Or even better, you can grab an image from *another* camera running in the room.
  * **Video Loop** Using `compsite` can we create a small video loop, of multiple cameras compositing images from each other.