Categories
IT Servers Streaming

Another Type of Re-Streamer Using Node.js

I came across another type of re-stream server that might help some people. It can also be run on a $3.50 per month server from Amazon Lightsail. It’s a node.js server called Node-Media-Server roughly according to this link: https://github.com/illuspas/Node-Media-Server. I’ve modified it slightly and made it useful for re-streaming to YouTube and Facebook.

The reason I searched out yet another re-streaming server type is that I found that streaming with Wirecast going to my other server under NGINX was unreliable. That server works very well with our hardware streamer and the streaming apps on iPhone and Android but it didn’t work well with Wirecast. The following server works very well with Wirecast as well as our hardware streamer and the apps that I have on my iPhone and Android phone.

My version here, is slightly different than the one on Github and I’ve laid out some instructions below. Please read the entire post before starting as we want to make sure it goes as smoothly as possible.

The first step in the process is to to to Amazon Lightsail. The website address is : https://aws.amazon.com/free/compute/lightsail/ They even have a one month free trial of their services for the smallest one. That’s the one we want so it’s great just to try for this. So even if you don’t know what you’re really doing, you aren’t even out anything to try.

Go to Get Started Now and begin. You’ll have to create an Amazon AWS account to start, but that’s easy. (I bet you’ll like it after the first month too) I can’t go back through this because I already have an account. I don’t think I can walk you through some of those details because of that.

However once you are through with that, you should get to a page that looks like this:

From here you choose your instance location. I always choose Northern Virginia for two reasons. One is that I’m located 40 miles from these server farms. Two is that these are probably some of the fastest server farms in the world.

Next you have to pick the OS for the server. Also you have to choose “OS Only” and I choose Linux with Ubuntu being the image I want. It runs the software amazingly and the instructions assume these choices. Now we have to name our “instance”. I like something like “re-streamer” or something of the sort.

Next you have to wait for it to be created, that takes about a minute or less. Once it’s up and running, we can log into it via the browser. That is very simple and you just click the hyperlinked name of your server. That brings you to a page with a huge button that says “Connect Using SSH”. That’s what we want.

Then it brings you to a command prompt. This is where we’ll issue all our commands to make the server do all our work.

Then you can begin with the following code at the command prompt. Press “y” when it asks and select “yes” by putting the red curser over it when you see a graphic ask a question. Or if it asks to press [enter] please do that.

My directions below are slightly different than the ones on Github but these solve some little issues you might find if you followed the others.

The below commands you should perform one at a time by cutting and pasting into the command prompt.

mkdir nms
cd nms
sudo apt-get install npm
npm install node-media-server
sudo apt install software-properties-common
sudo apt update
sudo add-apt-repository ppa:jonathonf/ffmpeg-4
sudo apt install ffmpeg
sudo nano app.js

When you use “sudo nano app.js” you’ll be opening a file. This is the main configuration file for the node-media-server. You’ll need to paste in the following code but put in any locations you want to send your stream to via RMTP. You have to replace my keys with your keys obviously.

const NodeMediaServer = require('node-media-server');
const config = {
  rtmp: {
    port: 1935,
    chunk_size: 60000,
    gop_cache: true,
    ping: 30,
    ping_timeout: 60
  },
  http: {
    port: 8000,
    allow_origin: '*'
  },

  relay: {
        ffmpeg: "/usr/bin/ffmpeg",
        tasks: [{
                app: "live",
                mode: "push",
                edge:"rtmp://127.0.0.1:1936/rtmp/<<YOUR FB KEY>>",
                appendName: false
                },
                {
                app: "live",
                mode: "push",
              
edge:"rtmp://a.rtmp.youtube.com/live2/<<YOUR YT KEY>>",
                appendName: false
                }
               ]
        }
  };
var nms = new NodeMediaServer(config)
nms.run();

We have to get stunnel so that facebook will work. We’ll go back to the command prompt from your server. Issue the following commands to get stunnel.

sudo apt-get install stunnel4 -y

Now we’ll have to change stunnel’s boot configuration, issue the following command:

sudo nano /etc/default/stunnel4

Change Enabled from 0 to 1. It should look like the following:

ENABLED=1

Next we have to edit the stunnel configuration file.

sudo nano /etc/stunnel/stunnel.conf

You’ll have to cut and paste this in its entirety. It should look like this:

pid = /var/run/stunnel4/stunnel.pid
output = /var/log/stunnel4/stunnel.log
setuid = stunnel4
setgid = stunnel4
# https://www.stunnel.org/faq.html
socket = r:TCP_NODELAY=1
socket = l:TCP_NODELAY=1
debug = 4
[fb-live]
client = yes
accept = 1936
connect = live-api-s.facebook.com:443
verifyChain = no

Then of course you’ll have to use ctrl-x to exit, and Y to save it as the original named file.

Next we have make it enabled after boot by doing the following:

sudo systemctl enable stunnel4.service

Now we have to restart stunnel because we changed the configuration files.

sudo systemctl restart stunnel4.service

Now that Stunnel is installed and the app.js file has been properly created we can try it out. To make it run you have to execute the app.js file. Use the following command. The & after the command will let it run in the background and won’t close when you close the command prompt

node app.js &

This should output something like the following:

Whenever you have to modify anything in that app.js file remember that it’s located in your home folder, under “nms”. So in linux you have to go to that location from login like this if you want to directly:

sudo nano ubuntu/nms/app.js

Then remember that to start it again you have to use the command “node app.js” or “node app.js &”.

Also if you have to stop the app.js from running you can press cntrl-x, which will exit the process and give you a command prompt again.

You’ll want to fix the firewall settings on the lightsail server instance so that our connections from outside can get to the server. You should allow ports, 8000,443, and 1935 along with the ports that are automatically allowed by lightsail. You can change this by going to the “networking” section of the instance. Then making it look like the picture below. You’ll also want to set the IP address of your streaming location so that other people can’t stream to your instance. the instance I created for this tutorial isn’t going to be running by the time you read this. I’ll be deleting it, so it will be safe.

Next you can test it with some simple software like VLC along with OBS or your churches streaming software or hardware encoder. You’ll need your RTMP feed to go to your server IP address which is seen on the home page of the instances of Lightsail. It’s on the bottom right above your server location. You can also set it to make it static at this point, but that will give it a new one that will not change. You’ll want that so you don’t always have to change your OBS or streaming software or hardware. To make a static IP just go to the instance by clicking the hyperlinked name and then choose networking from the options across the middle of the page. Then choose static IP on the left.

This gives a nice output page as well to see the status of your running server that’s waiting for your stream. Go to your IP address 000.000.000.000:8000/admin

In OBS or your hardware encoder choose these options in the stream settings:

Service: Custom
Server: rtmp://<<ngnix server IP address>>/live
Stream Key: test

For a playback test use VLC or something that can play RTMP streams. Set it up as follows: Go to Network Stream, Network.

network URL: rtmp://<<node-media-server IP address>>/live/test

You can also go see what the admin page looks like at that IP address:8000/admin, it will show you the inputs and outputs as well as memory usage and cpu usage.

Also now you should be able to go to Facebook and YouTube to see it there to go live with if you wish. Everything should be up and running at this point. You might want to take a snapshot here, because that way you can always fix anything else that goes wrong. Also if you want to further modify this for other purposes to make it better later you have a good jumping off point to create a new instance without jeopardizing your running one.

******The below section about PM2 is an addition to this post to make the program work better for streaming events without worrying about the terminal window closing or disconnecting.

To make this app.js function continue to work even if you close the command prompt, which you will probably want to do reliably. You should install another small program that will make this continue in the background and continue pushing your streams anytime you need them. We’re going to use PM2 to do this.

Install PM2 by typing the following at the command line:

sudo npm install pm2 -g

There are some huge benefits for you if you run your application using pm2. Make sure you are in the directory /nms/ where you installed the app.js file, and instead of running the app as we did above (with node app.js), we’ll run it now using the following command:

pm2 start app.js

This will show something like:

This PM2 kept our server up and running without any input for a 24 hour test we inadvertently ran yesterday through this morning.

If you have to re-do any of the app.js file you might have to restart the pm2 command, some of the below might be helpful as well in that case. If you have to restart the pm2 command, sometimes it will cause two running processes. You can check what’s running under pm2 processes by using the command pm2 list

pm2 list               # this lists the running processes
pm2 stop all           # Stop all processes
pm2 restart all        # Restart all processes

pm2 reload all         # Will 0s downtime reload (for NETWORKED apps)

pm2 stop 0             # Stop specific process id, the id is shown as a 0 in the examples here
pm2 restart 0          # Restart specific process id

pm2 delete 0           # Will remove process from pm2 list
pm2 delete all         # Will remove all processes from pm2 list

# Misc

pm2 reset <process>    # Reset meta data (restarted time...)
pm2 updatePM2          # Update in memory pm2
pm2 ping               # Ensure pm2 daemon has been launched

Also remember to type “exit” to get out of a terminal window safely. Don’t just close the window with the X on the top.

If you need more help, please email me tim@timstreams.com or find me on Facebook.

Categories
IT Servers Streaming

Simple Re-Stream Server Instructions

HLS and Re-stream server configuration using 2 separate servers

Note: If you use Wirecast for streaming, you might want to try my other re-streaming server instructions as our Mac version of Wirecast didn’t like this one much. It wasn’t perfectly reliable. Howevever our hardware streamer worked great with this one, and other people have great success with it with other streamers as well. The linked one works well with Wirecast. https://timstreams.com/another-type-of-re-streamer-using-node-js/

I have been asked a few times for some very simple to follow server creation instructions that people can follow to make a server that can take a single RTMP feed and send it along to multiple services. In this case I have personally used this to send a stream to Facebook and YouTube at the same time. It only requires going to the Facebook and YouTube pages and telling them to start the live post.

I just went through the following steps and created a new streaming server in about 20 minutes. It was very fast and I didn’t really think much about what I was doing. I just sort of blindly followed this. At the end I tested it with my Facebook page and it said it was receiving it and was ready for me to go live.

This process I describe uses Amazon Lightsail servers that are very cheap and easy to use. The server that’s needed is $3.50 per month. In the cases I’ve been asked about churches are trying to do this and I assume you would need this for something like Sunday and Wednesday night each week. This would use about 5GB of data each time, so about 10 to 20GB per week. Which means about 80GB per month for the input stream. If you send this to Facebook and YouTube you are also sending out 160GB per month out of the server. This is well within the 1TB per month that the Lightsail small server gives you included for your $3.50.

The first step in the process is to to to Amazon Lightsail. The website address is : https://aws.amazon.com/free/compute/lightsail/ They even have a one month free trial of their services for the smallest one. That’s the one we want so it’s great just to try for this. So even if you don’t know what you’re really doing, you aren’t even out anything to try.

This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-12.08.21-PM-1024x576.png
This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-12.10.44-PM-1024x576.png

Go to Get Started Now and begin. You’ll have to create an Amazon AWS account to start, but that’s easy. (I bet you’ll like it after the first month too) I can’t go back through this because I already have an account. I don’t think I can walk you through some of those details because of that.

However once you are through with that, you should get to a page that looks like this:

This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-11.13.09-AM-1024x576.png

From here you choose your instance location. I always choose Northern Virginia for two reasons. One is that I’m located 40 miles from these server farms. Two is that these are probably some of the fastest server farms in the world.

This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-11.13.16-AM-1024x576.png

Next you have to pick the OS for the server. Also you have to choose “OS Only” and I choose Linux with Ubuntu being the image I want. It runs the software amazingly and the instructions assume these choices. Now we have to name our “instance”. I like something like “re-streamer” or something of the sort.

Next you have to wait for it to be created, that takes about a minute or less. Once it’s up and running, we can log into it via the browser. That is very simple and you just click the hyperlinked name of your server. That brings you to a page with a huge button that says “Connect Using SSH”. That’s what we want.

Then it brings you to a command prompt. This is where we’ll issue all our commands to make the server do all our work.

The following directions mostly follow from the following page: https://sites.google.com/view/facebook-rtmp-to-rtmps/home

Now that we have a server created and the instance is running and we are in the command prompt.

The first step is to install Nginx and Stunnel. Stunnel is what’s needed to make Facebook work, it requires an RTMPS feed. If you don’t want Facebook to work, it’s even easier because you can leave a few steps out. We won’t go into that in this post because most people want to stream to Facebook and YouTube at least.

To install Nginx and Stunnel we do the following: (litterally) copy and paste these each individually into the command prompt and hit enter. Some of them will ask questions as you go, like do you want to continue, it will take some amount of disk space. You have to agree, but it is always small and we have 20 usable GB of disk space. We’re not concerned. You also will get a couple windows like in the pictures below, choose YES on them or choose the Top Choice. These windows with the images need to be selected with the arrow keys to highlight the choice and then use the enter key to choose it.

If while you’re using the command prompt you have to re-type the last thing you did before, you can just hit the up arrow key. It will retype the last command automatically. Also as a note in the command prompt the mouse can’t be used to move the cursor, you have to use the arrow keys.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install nginx -y
sudo apt-get install libnginx-mod-rtmp -y
This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-11.19.47-AM-1024x576.png
This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-11.21.36-AM-1024x576.png

This is totally optional: What I like to do occasionally is take a “snapshot” of the instance. This makes a copy of it’s drive and allows us to recreate it immediately if we mess anything up. The snapshot is like our backup plan if we really destroy this thing. It will save time if you have go try again. The only downside is that we pay a few cents a month to store each snapshot. You can delete any of them at any time.

Next we’ll edit the configuration file that runs Nginx. This file is what does most of the work that we need. The configuration is how we tell it what we want to do. Cut and paste the below into the command prompt.

sudo nano /etc/nginx/nginx.conf

At the very bottom of the nginx.conf file, under the other stuff that’s already there, paste this in in it’s entirety:

rtmp {
        server {
                listen 1935;
                chunk_size 4096;
                application live {
                        live on;
                        record off;
                        push rtmp://127.0.0.1:1936/rtmp/<<Facebook persistent stream key >>;
push rtmp://a.rtmp.youtube.com/live2/<<YouTube persistent stream key >>;
                }
        }
}

You’ll have to insert your Facebook persistant key in place of the note in this code. The key should go between the / and the ; This push command sends our stream and Facebook Key to stunnel so it can be made secure for Facebook to accept it. Below this “push” command we can put another in so we can send to YouTube or anywhere else we want. You can see an example from my personal nginx.conf file below. For YouTube you’ll have to add a push command that sends it to “rtmp://a.rtmp.youtube.com/live2/”. You’ll have to put your KEY after the last / and then put a ; at the very end of that line.

This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-1.13.27-PM.png
Insert your new part below the commented out parts that are already there.

After you’ve modified the nginx.conf file you have to save it and close it, so you press cntrl-x, then it will ask if you want to do that to the current named file and you hit the Y key, and then press enter for it to save the file.

If you need to keep other streamkeys and push commands in there when they aren’t needed, you can comment them out by using the # symbol at the start of the line. That makes the computer skip over that line and just go to the next one.

Now we have to restart Nginx because we’ve modified the configuration file. Paste in the following command into the command line.

sudo systemctl restart nginx

When you attempt to restart nginx, if it doesn’t work, you have errors in that nginx.conf file. You have to be sure it looks just like the examples or it won’t run properly and will fail. Be sure you have ;’s after the lines of the push commands.

If at any point you have to re-adjust the .conf files, you’ll need to restart nginx again.

Next you can test it with some simple software like VLC along with OBS or your churches streaming software or hardware encoder. You’ll need your RTMP feed to go to your server IP address which is seen on the home page of the instances of Lightsail. It’s on the bottom right above your server location. You can also set it to make it static at this point, but that will give it a new one that will not change. You’ll want that so you don’t always have to change your OBS or streaming software or hardware. To make a static IP just go to the instance by clicking the hyperlinked name and then choose networking from the options across the middle of the page. Then choose static IP on the left.

This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-1.22.42-PM.png
This image has an empty alt attribute; its file name is Screen-Shot-2020-05-01-at-1.25.47-PM.png

You should also set the firewall rules to allow port 1935. That’s the one that RTMP is on and we do that through the networking page under the IP information.

This image has an empty alt attribute; its file name is Screen-Shot-2020-05-03-at-7.40.06-AM.png

In OBS or your hardware encoder choose these options in the stream settings:

Service: Custom
Server: rtmp://<<ngnix server IP address>>/live
Stream Key: test

For a playback test use VLC or something that can play RTMP streams. Set it up as follows: Go to Network Stream, Network.

network URL: rtmp://<<ngnix server IP address>>/live/test

After we have that all running. We have to get stunnel so that facebook will work. We’ll go back to the command prompt from your server. Issue the following commands to get stunnel.

sudo apt-get install stunnel4 -y

Now we’ll have to change stunnel’s boot configuration, issue the following command:

sudo nano /etc/default/stunnel4

Change Enabled from 0 to 1. It should look like the following:

ENABLED=1

Next we have to edit the stunnel configuration file.

sudo nano /etc/stunnel/stunnel.conf

You’ll have to cut and paste this in its entirety. It should look like this:

pid = /var/run/stunnel4/stunnel.pid
output = /var/log/stunnel4/stunnel.log
setuid = stunnel4
setgid = stunnel4
# https://www.stunnel.org/faq.html
socket = r:TCP_NODELAY=1
socket = l:TCP_NODELAY=1
debug = 4
[fb-live]
client = yes
accept = 1936
connect = live-api-s.facebook.com:443
verifyChain = no

Then of course you’ll have to use ctrl-x to exit, and Y to save it as the original named file.

Next we have make it enabled after boot by doing the following:

sudo systemctl enable stunnel4.service

Now we have to restart stunnel because we changed the configuration files.

sudo systemctl restart stunnel4.service

Now since we changed the stunnel configuration and all we should restart NGINX for good measure.

sudo systemctl restart nginx

Everything should be up and running at this point. You might want to take a snapshot again here, because that way you can always fix anything else that goes wrong. Also if you want to further modify this for other purposes to make it better later you have a good jumping off point to create a new instance. 🙂

Because this is not limited to who can send a stream to it and it’s sitting exposed with port 1935 open, you might either want to delete the open port of 1935 in Lightsail when it’s not in use, or just shut it off when not in use. If you want to shut it off when not in use just issue the command “sudo systemctl stop nginx”. This way no one else can send it anything. When you want to start it again obviously issue “sudo systemctl restart nginx”. If you want to solve it a better way yet, you can find your IP address for your church and make it so that only people coming from your church’s IP address can stream to your server. You would have to add a few lines in the beginning of that rtmp section of the nginx.conf file. You add the “allow publish 000.000.000.000;” and “deny publish all;” below the “chunk_size 4096;” and above the “application live {“. You’ll have to put your locations IP address in that field where the zeros are in my example.

rtmp {

        server {

                listen 1935;
                chunk_size 4096;
                #below is IP for your streaming location
                allow publish 000.000.000.000;
                #below is so others can't use the server
                deny publish all;

                application live {

Remember that whenever you change the nginx.conf file you have to restart the service so that the changes take effect by using the command : sudo systemctl restart nginx

If you need more help, please email me tim@timstreams.com or find me on Facebook.

On about May 5th, As a note, I had my 12 year old run through this and try out my instructions. He’s never done anything like this at all. He is a gamer though and has used computers for years in schooling. It took him 1 hour and 2 minutes including when I made a mistake when setting up streamkeys and also with him giving me junk for minor spelling issues and unclear instructions. I’ve since fixed most of the points he helped me with. I’ll probably use his server to stream the Live church service Wednesday night.

Categories
IT Servers Streaming

Virtual Services and Streaming

This is what our server configuration looks like. This is a quick document that I made up to show the leadership what I did.

Because of our new uptick in streaming we had to figure out how to stream our live services with RTMP to Facebook and YouTube, as well as our app. The app is a little odd as it requires a stream input of a .m3u8 file, also called HLS. Our church cable modem is not able to support more than one 1080 x 30fps output at once. This is about a 4mps stream. So we needed something to help us.

This led us to research services like restream.io and Boxcast. Both these will do it for RTMP, however Boxcast will do that and also the .m3u8 (HLS). That brought up an interesting challenge in my mind. I figured that this couldn’t be “that” hard. I just had to figure out how to re-stream RTMP. That led me to cloud servers which I had a tiny bit of experience with after-all this website is on an AWS Lightsail cloud server which costs $3.50 and my other website timrides.com is on a free Google cloud server instance.

I had experience in spinning up cloud servers with Bitnami originally and running apache as a webserver. I had seen the wonders of what can be done, but didn’t know how to do more than run WordPress on those Bitnami installations for the websites.

So I went to Amazon and to the Lightsail section which is sort of the simple part of AWS where you pay for various size cloud servers and all your data and things are included. It makes billing simple and you don’t have to worry much about causing some huge bill. You can get there at https://lightsail.aws.amazon.com/ .

The first part of making the re-streaming server for RTMP is getting an instance such as the $3.50 one, it’s good for at least re-streaming to 3 places. So you take in your RTMP feed from your streaming software like wirecast or obs and you send it to the server. The server takes it and sends it on to the services like YouTube and Facebook. If you need more, you might need a bigger server like the $5 one. In any event its cheap and even if one is to small you can take a snapshot of it and spin up a bigger one in about 4 minutes. I chose to run Ubuntu 18.04 on mine. It’s well documented and easy to run NGINX on that to get a good webserver and robust re-streaming server all in one.

I did a ton of Google searching for information and came upon some reasonable tutorials. It was not very hard to get the re-streaming part of this going. Just forwarding streams of RTMP is not to hard and obviously doesn’t even take much computing power since the Ubuntu machine with 512mb of RAM can do it. It requires some simple config file modifications to the NGINX software with the rtmp module in it.

Also to get Facebook to work you have to add in a module called “stunnel”. That is pretty simple as well. The tutorial below should have most of what you need for Facebook, YouTube and any other service that requires only RTMP.

We’ve now streamed a few times to Facebook, YouTube and my HLS streaming server a few times. It works very well with this simple RTMP streaming server setup. It’s running well on my tiny $3.50 per month server just chugging along. Once the persistent stream keys are input in the server there isn’t any maintenance for it or any other reason to look at the configuration files. I’ll modify it sometime to add some of the statistics information that I have on my HLS server.

In addition to the below, you should fix the firewall to only allow through ports 1935, 80, maybe 440,990 and 1000. Depends on what else you do with it.

https://sites.google.com/view/facebook-rtmp-to-rtmps/home

If you’d like to see some of my nginx.conf file and other parts needed for this project see below. I’ll insert them here.

If you need to output an HLS or .m3u8 file, you need to go a little further. I made two servers so that I could get the majority working quickly and then I could modify the other without messing up the working one.

My second server needs to be made bigger to carry the load when we stream to the app. The app users are directly receiving the file from my server and they put a load on it. Also the creation of that HLS file is intensive alone, even if only one person is on the app. I’ll post about that second server in another post.

I hope this brings you as much of a feeling of accomplishment as it brought me. It also taught me a ton about linux and cloud servers.