Connected Devices and Networked Interaction
Week 1: Serial to Browser Device
For the first job I need to build a microcontroller device and browser-based application using the HTML DOM elements. Send data from the device only when one of the sensors changes.
I decided to start with the construction of the website, here I used my professor Tom Igoe’s example as the core device, and also used switch and potentialmeter as my analog and digtal input.
Here I have linked a switch and potentialmeter, and I have added two LEDs to indicate the current state of the switch on the physical side, helping me to better catch errors in later stages. And I also hope that in the next attempt, I can use the computer to control the brightness of the light.
Next, I tried to connet the physical device to the computer, but unfortunately after several attempts, it failed. The reason is that the computer side cannot receive the data from the physical side, but strangely, there is no problem with the code and connection. So I tried to change the automatic port selection to a fixed port. After debugging, it worked as expected.
After the user presses the button, the web page can reflect the state of the button well, and the LED display results can also be well matched. Web pages can also display the degree of potentialmeter. (I added a tiger logo because today is Chinese New Year)
Question of the week: Why when I use auto port selection, I get two identical ports and the web page doesn't accept readings.
Solution: In Windows I can change the port name and make the port open,
Week 2: QR Code Application
For this week I tried to generate the QR code online and display the code by using OLED screen, the screen I used is: Adafruit 1306. To see if my screen is correctly connected to the computer and is working properly, I have it show the time and the state of a sensor attached to my controller.
Next, I uploaded my own QR code to the Arduino and set its resolution to 128×64 to get the best display. And it will link to the website: https://ezivis.github.io/zeshuqr.github.io/
Week 3: HID-based game controller
This week I will make a Human Interface Devices (HID) devices from a microcontroller, i.e., Keyboard and Mouse devices. As a result, I will be able to use the HID as a controller to play the Lunar Lander game. First of all, I tried to add one joystick controller to control the mouse and test whether this works well, but after I completed the circuit connection and coding, I found my console could get the data, but my mouse won’t move. Soon I realized it was windows system issue:
After updating the driver from the Arduino device to the USB input device, I successfully control the mouse by using the joystick, and it works very well. Since the input data is not as stable as I expected, I set the threshold to 2 to stabilize the controller stabilized.
After this, I decided to add another joystick to control keyboard input. The basic logic is the same as the mouse. The difference is that I subtly use the if statement to convert the numbers into the input text. The keyboard will input different content when the x-axis or the y-axis is greater or less than a value. The result is as I expected it to be stable and controllable. (Although I still don’t know how to complete the game tasks as expected)
Week 4-7: Midterm Project: Sky Light Clock
Concept
Whenever I feel stressed, I always walk to Brooklyn Bridge Park to catch the sunset to take a breathe from my busy school schedule. Many studies have shown that exposure to nature views such as sky and trees has a positive effects on mental health, reducing stress level and helping anyone view their situations in a larger context. In our busy urban lives, we all need some outdoor views to give ourselves a moment of break. To bring nature to the indoor space, I plan to build a LED sky light clock that can imitate the sky colors to bring calm and pleasure to a device that is often seen as a source of stress.
This project is inspired by the reflective shadow clock form Breaded Escalope studio and my favorite Triwa cloud watch. Both projects use the ephemeral quality of light to present new sensory experiences with time.
Vienna design collective Breaded Escalope has created a clock that uses shadows to display the time when a finger is placed in its face.
This Limited Edition Triwa watch challenges the concept of time through hazy dials & colorful, cloud-like watch hands to represent the transient moment of dawn.
Interaction/Score
The clock is designed to be simplistic to focus on the lighting effect. the base color of the clock is made of the gradient blend between minute and hour hand color, and the second hand color will move across the screen slowly. To make sure the color across the clock can blend smoothly together, size of the clock is a medium-sized so that light emitted from leds can reach beyond the center point.
One of the early ideas is to extract color from the sky image as a ring and project it to led. This idea could potentially create beautiful gradients, but it would not help serve the function of a clock.
Sky ring translation
To change the color of the LED, user can access the customization panel by scanning a QR code, and upload pictures of sky to the online gallery. User can use color pickers to choose three colors from the selected image, and the colors will be updated to the clock hands via BLE.
User can view the BLE connection on the top right corner, the clock will blink as well to indicate successful connection. By introducing a digital component, we aim to make this a collective experience that can be shared with others, and to remind people that we are all under the same sky.
(Unfortunately, due to web BLE service limitation, access from phone is limited to android users)
Process
Bill of Materials
Technical Requirements
- Physical – 3D modeling, CNC Routing, Arduino fastLED, Arduino BLE
- Digital – html, css, BLE.js
Fabrication
2/25/2022 – First-round fabrication
Fusion 360 → MasterCAM → CNC
Meijie learnt how to operate CNC machine with Aiden and this page (and now I prefer CNC so much more than laser cutter for woodwork…) There was a miscalculation for the inner diameter that made the ring smaller than the length of the 120-leds strip, so I had to remodel and CNC another one.
Gamma Correction
Helpful Reference for creating gradient
Conversation with lighting artist Erwin Redl –http://paramedia.net/externalpages/NYCTheHole.php
how do you get the gradient?
thin board to block between outer/inner led strip, 16-bit color palette,
control the RGBW independently for each led strip,
more color doesn’t mean better, blue red yellow
Gamma correction
2/28/2022 – light color test with Gamma correction
In order to get the light color to reflect the color we see on the screen, we added gamma correction to the rgb values we got from the color picker to make lighting brightness correctly reflect the brightness of the display color.
Website
To create the website, we needed to provide the user with a timer, Bluetooth, upload image and color picker function. So we chose to use javascript and jquery to implement most of the features.
Since the others are fundamental programming points, I would like to share more about Arduino BLE in javascript here.
Although p5.ble.js is an API designed specifically for p5.js, we can still use most of its features without applying p5.js. However, since Arduino can only accept an array of data, I defined the data obtained by the user as the global variables in the color extraction part to facilitate the data transfer.
Then the obtained data will be transformed into a string of arrays by Unit8Array in function and transferred to Arduino. The specific splitting will be done in Arduino part.
Arduino programming
3/1/2022 – Real-time time update, real-time color update, used the internal rtc module
System Diagram
Schematics
Some issues we encountered:
- We switched from animating the light using 3 nest-in for loops (takes 1h to finish a loop) to animating with rtc time update (very fast looping), which allows us to update color in real time.
- To prevent from updating the LEDs too frequently, we only update the LEDs when we detect a change in values in rtc values.
while (central.connected()) can make it hard for the clock to keep running when no peripheral is connected to it and execute actions at the moments of connection/disconnection. Instead, we used event handlers to listen to different events, which makes the code much simpler and easier to execute a wide ranges of actions.
We added time calibration button on the web since internal rtc only update to correct time when code is compiled to the Arduino. This makes it easier to recalibrate as long as BLE is successfully connected.
Final fabrication
Meijie made the inner diameter of the front ring small to better hide the led strips, and widened the inner diameter of the back rings to just fit the length of the leds. We also minimized the connections by removing unnecessary components (external RTC, capacitors, resistors) and fit the wires into a small chip. As shown in the image, I adjusted the electronics pocket into an asymmetrical shape so that it is slightly easier for wire to plug into the Arduino board.
Back view
Week 8-9: WiFi connectivity and Websockets and MQTT
For the connection between the Arduino Nano 33 IOT and WiFi, I first tried to check the Wi-Fi status to ensure that my WiFi was active and that my Nano 33 IOT could search for WiFi.
After making sure the Arduino’s WiFi function runs OK, I added arduino_secrets.h and included SECRET_SSID, SECRET_PASS respectively. Opening Serial shows that the Arduino is connected to WiFi and can get web information from www.example.com.
Next, I’m going to create an html page myself using localhost. To do this I need to change the web link in Arduino to my computer’s IP Address and change the generic port:80 to my custom port:3000, then install the files I need on the computer side via npm install and npm install express, and then run my node server.js. We can find that the Arduino is receiving my custom URLs very well.
Question: Is there any way to send data (similar to BLE) from a web page to an Arduino while using a WebSocket. And can it always be connected to the network to receive data?
Next, I linked the Arduino to MQTT via the MQTT tutorial and could see that when I pressed the button, I would receive a subscription message from the broker and my LED would also light up to indicate that I had received the message.
Question: Can I customize the MQTT service, and can I link MQTT to the web pages I create?