Live Web
Week 1: What is the "live" web?
So the last live web I’ve consumed is google web meeting, and the reason I consumed it is simply because my professor/supervisor asked me to, so I had to. I think the positive impact on me is that I don’t have to feel the pressure of a lot of body language because of face-to-face meetings, and I can be more relaxed in a familiar environment. The negative effect is that online meeting is not as efficient as in reality, and it takes a lot of time to understand each other. I think in the process, I lost a lot of opportunities to meet people face-to-face and truly felt the meaning and feelings they wanted to express.
Week 2: Serverside
For the second assignment, we needed to get up and run with Node.js on Digital Ocean and try to make a chat application. We planned to make a collaborative diary by connecting users and working together to create a diary with features.
Users can click on generate to generate random words and write their own paragraphs based on the rules. We added 60 interesting words to the database to give users more ideas to create more interesting paragraphs.
When users finish uploading their sentences, everyone’s page will receive the same message, much like chat software. And as more and more users complete it, our diary will get better and more interesting.
Week 3: Transmitting Other Data Types
This week, I will improve on the project, sending a different type of event to the server and the clients. To achieve this goal, I decided to create a tug-of-war game. Users will be able to interact with their opponents and receive results in real-time. They will also be able to see the status of each other’s mouse in real-time. What’s more interesting is that the game is open-ended, which means we have the opportunity to have a 10 vs. 10 tug-of-war game.
Week 4 and 5: Transmitting and Saving Data from Multiuser Audio/Video
Since I need the site to be able to stream video and audio, I need to get up and running with HTTPS, because only through this “secure” version of HTTP can the web page give us access to the user’s camera and microphone.
After having access to the camera and microphone, I decided to make a less harmful “scam site” since I was asked this week to be able to save users’ data. I wanted to make a site that would appeal to users to protect their privacy more than the standard camera feature. The goal was to alert users to sites that steal personal information with simple features.
I first created this cute-looking cover with the purpose of letting down the defenses of users who see it. And this seemingly simple feature will also give users more confidence that this is a site that will help them change their face style.
When the user clicks convert now, the interface will jump to the results screen, the user will quickly realize that they have been scammed. The new website can show the front to get the user’s photo information and get the user’s geographical location and time of operation. This is undoubtedly a significant privacy breach.
And I convert the image to a URL string and save it in my server’s database by using the command “const image64 = video.canvas.toDataURL();” so that the user has no right and no way to access the data I save.
I hope that users will not be easily victimized by similar websites or applets online after this, as a way to improve people’s security knowledge.
Midterm Project: Collaborative Nose Painting
Since COVID-19, our lives have changed dramatically, and people have become more and more separate, and I think this project will be an opportunity for us to help people get together again through the live web.
For the midterm project, me and Lifei decided to team up and created a collaborative drawing game on people’s video.
There will be a 30 seconds count down, and during this period, the website will display a random participant’s camera. After 30 seconds, it will take a picture of the painted video.
We were thinking to use facemesh to track a few key points on a face and draw with those points. P5.js
We wanted to add joy to the user than the boring tracking type decoration, so we finally decided that the core interaction of our game is to let the user move their nose to paint. The reason for choosing the nose is that we need to cover our nose during the pandemic, and drawing at home with the nose is a reflection of bridging the gap.
To achieve our goal, we need to prepare two pages: the first one is to share our video and make features that can track the user’s facial details. At the same time, we want the user to generate different words randomly to increase the fun of the game. When the user reselects a word, the game timer will be reset, thus making the game more complex and challenging.
After we implemented single user player, we felt it wasn’t enough because it didn’t connect people to people, so next, we decided to make it multiplayer, and players could share game progress and data in real-time.
Next, we decided to add players. We resolved that players could share the coordinates of facial recognition in real-time via the web socket and package the coordinates to reach a real-time online drawing experience.
We would have liked to allow users to share coordinates and video streaming for the best experience. However, after user testing, many users were uncomfortable with video sharing on other people’s pages, so we finally decided to share only the brush but not the video streaming.
Week8: Video chat application in 3D (Three.js) and p5livemedia
This week we had a lecture on Three.js with our guest lecturer Aidan Nelson. And I decided to use what I learned in class to create a 3D video chat application.
I first got the template from Aiden’s GitHub, but since it can only run in the localhost, I thought it could run on my server to implement a video chat for everyone who enters my link, so I decided to modify server.js.
I removed the localhost code and added WebRTC also HTTPS to allow users to use webcam and audio. So that when I run server.js, users will be able to network through my link for the 3D web video chat.
Next, I tried to modify the 3D interface, such as adding a skybox or adding a model, but unfortunately, I still don’t understand how Aiden does the server data transfer, and when I tried to add a skybox, it crashed. I think it’s because I want to modify the operation, but Aiden has already defined it, so I need to alter a lot of data to achieve it.
The use of p5livemedia is much simpler because, unlike Peer to Peer Data and socket.io, p5livemedia does not require a lot of code in server.js. Instead, it can be implemented in a few simple lines. So I define the link and the app.get. Streaming video and data transfer will be done in p5.js.
Week9: Utilizes MediaRecorder and NeDB.
For this assignment, I decided to create a scene-sharing video where users can organize meetings in virtual scenes and can record the corresponding content or become the host.
I first made a human version, the middle connect button is for now video and saved to nedb, but later I can also add it as a moderator function. The saved file is temporarily stored in ram before refreshing, but the actual file is also saved to the database.
Next I added SimplePeer so that users could join and have live video. Not only that, but users can also save and show the video recording by clicking the “connect” button at the bottom of the video.
Then I record each user who enters the room by peer.length and number them so that each video has a unique class, which means I can implement various functions by different arrangement of classes. In the later course, I will add for loop to let each user’s video can be DISPLAYED and made into a monitor room effect.
