Nodester simplified auto-installer needed #8

akumpf · 2012-05-31T22:34:01Z

These couple of scripts are great, but I'm having issues getting it to work....

Ideally, the script would turn a fresh server install (Ubuntu 12.04/11.10 is what I'm trying to get running) into a Nodester machine! I think it's okay if it's initially pretty vanilla and default (default usernames, passwords, etc), as long as the script points to where things should be updated to make it secure.

So with that in mind, I've been chipping away at a full soup-to-nuts auto-installer. Ideally you just copy over one .sh file, run it, and watch as your server turns into a Nodester powerhouse :)

Here's what I have so far (NOTE: it doesn't fully get things up and running yet!)

echo "###########################################################";
echo "# STARTING INSTALL PROCESS >> Press Ctrl+C now to Cancel! #";
echo "###########################################################";
sleep 5;

echo ".----------------------------------------------------------.";
echo "| Fetching nodester installers                             |";
echo "'----------------------------------------------------------'";
sleep 1;
mkdir ~/nodester-installer
cd ~/nodester-installer
wget https://raw.github.com/nodester/nodester-installer/master/dependencies_verify.sh
wget https://raw.github.com/nodester/nodester-installer/master/env_creation.sh
wget https://raw.github.com/nodester/nodester-installer/master/package.json
wget https://raw.github.com/nodester/nodester-installer/master/nodester_official_install_guide.txt
chmod a+x *.sh

echo ".----------------------------------------------------------.";
echo "| Updating Apt-Get                                         |";
echo "'----------------------------------------------------------'";
sleep 1;
apt-get update

echo ".----------------------------------------------------------.";
echo "| Removing Apache (it may be installed by default)         |";
echo "'----------------------------------------------------------'";
sleep 1;
apt-get remove apache2
update-rc.d -f apache2 remove

echo ".----------------------------------------------------------.";
echo "| Installing apt-get packages                              |";
echo "'----------------------------------------------------------'";
sleep 1;
apt-get install git-core curl build-essential openssl libssl-dev psmisc couchdb htop 

echo ".----------------------------------------------------------.";
echo "| Nodester: Create CouchDB Nodester account                |";
echo "'----------------------------------------------------------'";
sleep 1;
couchdb -k;
sleep 1;
if [ `grep -Fc "nodester" /etc/couchdb/local.ini` != "0" ]
then
    # code if found
    echo "No need to update /etc/couchdb/local.ini, Looks like nodester change was already made."
else
    # code if not found
    echo "adding nodester admin account [pw=password] to CouchDB";
    echo "nodester = password" >> /etc/couchdb/local.ini
    /etc/init.d/couchdb stop
    sleep 1;
    /etc/init.d/couchdb start
    sleep 1;
    couchdb
fi
couchdb -b

echo ".----------------------------------------------------------.";
echo "| Setting up directories                                   |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/
mkdir build

echo ".----------------------------------------------------------.";
echo "| Node: Get latest stable version via Git                  |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/build
git clone https://github.com/joyent/node.git && cd node
git checkout v0.6.17
#git checkout v0.4.9

echo ".----------------------------------------------------------.";
echo "| Node: Configure, build, & install                        |";
echo "'----------------------------------------------------------'";
sleep 1;
./configure
make -j2
make install
node --version
ln -s /usr/local/bin/node* /usr/bin/

echo ".----------------------------------------------------------.";
echo "| INSTALL NODEJS & DEPENDENCIES...                         |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/nodester-installer
./dependencies_verify.sh 

echo ".----------------------------------------------------------.";
echo "| INSTALLING NODESTER -- :]                                |";
echo "'----------------------------------------------------------'";
sleep 2;
sudo ./env_creation.sh

echo ".----------------------------------------------------------.";
echo "| ENSURE NPM DEPENDENCIES ARE MET                          |";
echo "'----------------------------------------------------------'";
sleep 2;
cd ~/nodester/nodester
npm update
npm update
npm update

clear;
echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Setup Nodester!                          |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/nodester/nodester
echo "1. Copy ~/nodester/nodester/example_config.js to ~/nodester/nodester/config.js";
sleep 3;
cp ./example_config.js ./config.js
#Edit the settings in nodester/config.js
nano ./config.js
clear;
echo "2. Copy scripts/example_gitrepoclone.sh to scripts/gitrepoclone.sh and update it with the key you specified in config.js.";
sleep 3;
cp ./scripts/example_gitrepoclone.sh ./scripts/gitrepoclone.sh
#Update scripts/gitrepoclone.sh with the key you specified in config.js
nano ./scripts/gitrepoclone.sh


echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Setup CouchDB Tables                     |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/nodester/nodester
./scripts/couchdb/create_all_couchdb_tables.js
./scripts/couchdb/setup_default_views.js


echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Starting up Proxy and Main API           |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/nodester/nodester
./bin/app_start.sh
sudo ./bin/proxy_start.sh

echo ".----------------------------------------------------------.";
echo "| All done :)   Test web frontent on port 80!              |";
echo "'----------------------------------------------------------'";
sleep 1;
wget -O /dev/null 127.0.0.1

So I'm sure there are a few issues to sort out here (database seems to not be fully setup, and ./bin/proxy_start.sh doesn't run), but this would be awesome to get ironed out.

I'd love to get the full Nodester platform up on my machine (and have a feeling that it will travel quickly if we can make it dead simple to install).

Any thoughts, suggestions, modifications, etc. would be great!

Cheers!
Adam

The text was updated successfully, but these errors were encountered:

chrismatthieu · 2012-05-31T22:50:22Z

I totally agree with you! The best people to consult on this would be @gradus and @AlejandroMG on Github.

alejandro · 2012-06-01T02:11:14Z

Cool! Just a few issues, first we are using n to manage multiple node.js versions. So this part should be:

echo ".----------------------------------------------------------.";
echo "| Node: Installing n (the node.js version manage)  |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/build
git clone https://github.com/visionmedia/n.git
cd n && make install

Edit: Sorry submitted by mistake

Well, after that you can install node.js:

echo ".----------------------------------------------------------.";
echo "| Node: Installing node |";
echo "'----------------------------------------------------------'";
sleep 1;
n stable

# more awesomeness

Also the couchdb ppa is very old, so we should change that ;)

And finally, the scripts to start the app are now upstart scripts, so the process just should copy then to /etc/init

There is one thing left still, the chroot envs, but that's another history.

But as i said above, this idea is awesome :-)

akumpf · 2012-06-01T06:20:48Z

Thanks for the info; that's super helpful to get pointed in the right directions.

I can tell it's getting pretty close, but i think there are some directory issues. Where does nodester want to live on the filesystem? I've seen references to:
/root/nodester/nodester
/var/nodester/nodester
/node/nodester/nodester

Those directory issues seem to be keeping the upstart script from running (I created a symbolic link to get around it, but stopped myself from going link crazy as that might pose some security issues and how things are sandboxed).

However, after the install you can start the proxy manually (goto ~/nodester/nodester/proxy and run ./proxy.js) and it will show up at 127.0.0.1 :)

So anyway, I've reworked it a bit with your suggestions and here's where it stands:

echo "###########################################################";
echo "#     -- STARTING NODESTER AUTO-INSTALL PROCESS --        #";
echo "#     >> Press Ctrl+C now if you want to Cancel <<        #";
echo "###########################################################";
sleep 5;

echo ".----------------------------------------------------------.";
echo "| Fetching nodester installers                             |";
echo "'----------------------------------------------------------'";
sleep 1;
mkdir ~/nodester-installer
cd ~/nodester-installer
wget https://raw.github.com/nodester/nodester-installer/master/dependencies_verify.sh
wget https://raw.github.com/nodester/nodester-installer/master/env_creation.sh
wget https://raw.github.com/nodester/nodester-installer/master/package.json
wget https://raw.github.com/nodester/nodester-installer/master/nodester_official_install_guide.txt
chmod a+x *.sh

echo ".----------------------------------------------------------.";
echo "| Updating Apt-Get                                         |";
echo "'----------------------------------------------------------'";
sleep 1;
apt-get update

echo ".----------------------------------------------------------.";
echo "| Removing Apache (it may be installed by default)         |";
echo "'----------------------------------------------------------'";
sleep 1;
apt-get remove apache2
update-rc.d -f apache2 remove

echo ".----------------------------------------------------------.";
echo "| Installing apt-get packages                              |";
echo "'----------------------------------------------------------'";
sleep 1;
apt-get install git-core curl build-essential openssl libssl-dev psmisc couchdb htop 

echo ".----------------------------------------------------------.";
echo "| Nodester: Create CouchDB Nodester account                |";
echo "'----------------------------------------------------------'";
sleep 1;
couchdb -k;
sleep 1;
if [ `grep -Fc "nodester" /etc/couchdb/local.ini` != "0" ]
then
    # code if found
    echo "No need to update /etc/couchdb/local.ini, Looks like nodester change was already made."
else
    # code if not found
    echo "adding nodester admin account [pw=password] to CouchDB";
    echo "nodester = password" >> /etc/couchdb/local.ini
    /etc/init.d/couchdb stop
    sleep 1;
    /etc/init.d/couchdb start
    sleep 1;
    couchdb
fi
couchdb -b

echo ".----------------------------------------------------------.";
echo "| Setting up directories                                   |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/
mkdir build

echo ".----------------------------------------------------------.";
echo "| Node: Installing n (the node.js version manager)         |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/build
git clone https://github.com/visionmedia/n.git
cd n && make install

echo ".----------------------------------------------------------.";
echo "| Node: Installing node                                    |";
echo "'----------------------------------------------------------'";
sleep 1;
n stable

echo ".----------------------------------------------------------.";
echo "| INSTALL NODEJS & DEPENDENCIES...                         |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/nodester-installer
./dependencies_verify.sh 

echo ".----------------------------------------------------------.";
echo "| INSTALLING NODESTER -- :]                                |";
echo "'----------------------------------------------------------'";
sleep 2;
sudo ./env_creation.sh

echo ".----------------------------------------------------------.";
echo "| ENSURE NPM DEPENDENCIES ARE MET                          |";
echo "'----------------------------------------------------------'";
sleep 2;
cd ~/nodester/nodester
npm update
npm update
npm update

clear;
echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Setup Nodester!                          |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/nodester/nodester
echo "1. Copy ~/nodester/nodester/example_config.js to ~/nodester/nodester/config.js";
sleep 3;
cp ./example_config.js ./config.js
#Edit the settings in nodester/config.js
nano ./config.js
clear;
echo "2. Copy scripts/example_gitrepoclone.sh to scripts/gitrepoclone.sh and update it with the key you specified in config.js.";
sleep 3;
cp ./scripts/example_gitrepoclone.sh ./scripts/gitrepoclone.sh
#Update scripts/gitrepoclone.sh with the key you specified in config.js
nano ./scripts/gitrepoclone.sh

echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Setup CouchDB Tables                     |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/nodester/nodester
./scripts/couchdb/create_all_couchdb_tables.js
./scripts/couchdb/setup_default_views.js

echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Copy over Upstart scripts                |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/nodester/nodester
cp ./upstart/* /etc/init/

echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Creating symbolic link in /var           |";
echo "'----------------------------------------------------------'";
sleep 1;
cd /var
ln -s ~/nodester/nodester nodester

echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Starting up Proxy and Main API           |";
echo "'----------------------------------------------------------'";
sleep 1;
cd ~/
start proxy
start app

echo ".----------------------------------------------------------.";
echo "| All done :)   Test web frontent on port 80!              |";
echo "'----------------------------------------------------------'";
sleep 1;
wget -O /dev/null 127.0.0.1:80

Note that during the install process you'll probably want to have at least 1GB of memory (tried it with a tiny 256MB instance for fun and the node.js compile stage slowed to a crawl, pushing everything to swap space).

More thoughts? It'd be great to know where you think Nodester best wants to live on the filesystem.

Adam

akumpf · 2012-06-01T18:57:17Z

Update -- it's working (and in color)! :)

That said, there are a couple small annoyances that would be great to resolve.

after the nodester user is added to the database (appending the line nodester = password to /etc/couchdb/local.ini), a full system reboot is need to get the user to be authorized for login -- i'm sure there's a simple fix here somewhere?
the n tool has a linking issue with npm if it's installed first. so, I added a line to force a symlink which seems to resolve it (but slightly non-ideal).
what should the next steps be after the install is complete? and how many of those can we automate or script?

Sorry for the big paste inline here: once it's a little cleaner, it'd be great to add it to this project nodester-installer

Cheers,
Adam

#!/bin/bash

# color used for printing
use_color=true
if ($use_color) ;
then
    BLDYEL=$(tput bold ; tput setaf 3)
    BLDVIO=$(tput bold ; tput setaf 5)
    BLDCYA=$(tput bold ; tput setaf 6)
    BLDRED=$(tput bold ; tput setaf 1)
    BLDGRN=$(tput bold ; tput setaf 2)
    NOCOLR=$(tput sgr0)
fi

echo ${BLDCYA}
echo "                     _           _"
echo "                    | |         | |"
echo "     _ __   ___   __| | ___  ___| |_ ___ _ __"
echo "    | '_ \ / _ \ / _' |/ _ \/ __| __/ _ \ '__|"
echo "    | | | | (_) | (_| |  __/\__ \ ||  __/ |"
echo "    |_| |_|\___/ \__,_|\___||___/\__\___|_|"
echo ${NOCOLR}

echo "###########################################################";
echo "#     -- STARTING NODESTER AUTO-INSTALL PROCESS --        #";
echo "#     ${BLDYEL}>> This should be run on a fresh install. <<${NOCOLR}       #";
echo "#     ${BLDYEL}>> It may destroy things you care about   <<${NOCOLR}       #";
echo "#     ${BLDYEL}>> so proceed only if everything id safe! <<${NOCOLR}       #";
echo "###########################################################";
echo "Type '${BLDGRN}yes${NOCOLR}' if you want to install Nodester (or just press enter to exit)!";
read keep_going
if [ "$keep_going" != 'yes' ] ;
then
    echo "----------------------------------------------------------";
    echo "   ok. no worries, then.       rock out!  \m/             ";
    echo "----------------------------------------------------------";
    exit;
fi

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Fetching nodester installers                             |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
mkdir ~/build
mkdir ~/build/nodester-installer
cd ~/build/nodester-installer
wget https://raw.github.com/nodester/nodester-installer/master/dependencies_verify.sh
wget https://raw.github.com/nodester/nodester-installer/master/env_creation.sh
wget https://raw.github.com/nodester/nodester-installer/master/package.json
wget https://raw.github.com/nodester/nodester-installer/master/nodester_official_install_guide.txt
chmod a+x *.sh

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Updating Apt-Get                                         |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
apt-get update

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Removing Apache (it may be installed by default)         |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
apt-get remove apache2
update-rc.d -f apache2 remove

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Installing apt-get packages                              |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
apt-get install git-core curl build-essential openssl libssl-dev psmisc couchdb htop 

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Nodester: Create CouchDB Nodester account                |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
#couchdb -k;
/etc/init.d/couchdb start
sleep 1;
if [ `grep -Fc "nodester" /etc/couchdb/local.ini` != "0" ]
then
    # code if found
    echo "No need to update /etc/couchdb/local.ini, Looks like nodester change was already made."
else
    # code if not found
    echo "adding nodester admin account [nodester:password] to CouchDB";
    echo "nodester = password" >> /etc/couchdb/local.ini
    /etc/init.d/couchdb stop
    sleep 1;
    nano /etc/couchdb/local.ini
    /etc/init.d/couchdb start
    sleep 1;
    #couchdb
fi
/etc/init.d/couchdb status
#couchdb -b
sleep 1;

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| VERIFY: Is there a nodester admin account for CouchDB?   |";
echo "'----------------------------------------------------------'";
echo ${BLDVIO}
curl http://nodester:[email protected]:5984
sleep 1;
echo ${NOCOLR}
echo "Type '${BLDGRN}yes${NOCOLR}' if the response looks good. Otherwise, press ENTER to exit.";
read keep_going
if [ "$keep_going" != 'yes' ] ;
then
    echo "${BLDYEL}----------------------------------------------------------";
    echo "ARG. sometimes the solution is to just restart the server.";
    echo "perhaps 'reboot', log back in, and try running this again.";
    echo "----------------------------------------------------------${NOCOLR}";
    echo "   - stay strong, young Nodester!  \m/                    ";
    echo "";    
    exit;
fi

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Node: Installing n (the node.js version manager)         |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd ~/build
git clone https://github.com/visionmedia/n.git
cd n && make install

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Node: Installing node                                    |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
n stable

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Node: NPM fix...                                         |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd /usr/local/bin
rm npm
ln -s /usr/local/n/current/bin/npm npm

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| INSTALL NODEJS & DEPENDENCIES...                         |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd ~/build/nodester-installer
./dependencies_verify.sh 

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| INSTALLING NODESTER -- :]                                |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
sudo ./env_creation.sh

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| ENSURE NPM DEPENDENCIES ARE MET                          |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd ~/nodester/nodester
npm update
npm update
npm update

clear;
echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Setup Nodester!                          |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd ~/nodester/nodester
#echo "-------------------------------------------------------------------------------";
#echo "1. Copy ~/nodester/nodester/example_config.js to ~/nodester/nodester/config.js";
#echo "-------------------------------------------------------------------------------";
#sleep 3;
cp ./example_config.js ./config.js
#Edit the settings in nodester/config.js
#nano ./config.js
#clear;
#echo "-------------------------------------------------------------------------------";
#echo "2. Copy scripts/example_gitrepoclone.sh to scripts/gitrepoclone.sh and update it with the key you specified in config.js.";
#echo "-------------------------------------------------------------------------------";
#sleep 3;
cp ./scripts/example_gitrepoclone.sh ./scripts/gitrepoclone.sh
#Update scripts/gitrepoclone.sh with the key you specified in config.js
#nano ./scripts/gitrepoclone.sh

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Setup CouchDB Tables                     |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd ~/nodester/nodester
./scripts/couchdb/create_all_couchdb_tables.js
./scripts/couchdb/setup_default_views.js

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Copy over Upstart scripts                |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd ~/nodester/nodester
cp ./upstart/* /etc/init/

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Moving to /node                          |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
mkdir /node
mkdir /node/logs
mv ~/nodester /node/

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Creating symbolic links in /var          |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd /var
ln -s /node/nodester/nodester nodester

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| NodesterHelper: Starting up Proxy and Main API           |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
cd ~/
start proxy
start app

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| All done :)   Test web frontent on port 80!              |";
echo "'----------------------------------------------------------'";
echo ${NOCOLR}
sleep 1;
echo "  >> SERVER RESPONSE IS BELOW  :)";
echo "     Note: it may look like a 404 because no apps are running yet.";
echo ${BLDVIO}
curl http://127.0.0.1:80
echo ${NOCOLR}

echo ${BLDCYA};
echo ".----------------------------------------------------------.";
echo "| Nodester Auto-Installation is Complete!      \m/         |";
echo "'----------------------------------------------------------'";
echo " \   Here are some next steps you should take...          / ";
echo "  |  1. get familiar with /node                          |";
echo "  |  2. view logs in /node/logs                          |";
echo "  |  3. change the password for user 'nodester'          |";
echo "  |  4. change the password for your database            |";
echo "  |  5. setup a domain for your server                   |";
echo "  '------------------------------------------------------'";
echo ${NOCOLR}

akumpf · 2012-06-01T19:00:14Z

oh, and total time from start to finish (even with the annoying reboot) is just 7 minutes to get Nodester fully up and running on a fresh install of Ubuntu 12.04!

alejandro · 2012-06-01T21:22:51Z

@akumpf you are awesome :-)! About the address I go with /node/nodester/nodester since it's a path with full control for us. Edit. Doh! Let me check this, if we can move (delete?) all the current scripts to a better place, or if you can make the PR that'd be awesome

akumpf · 2012-06-01T22:41:40Z

@AlejandroMG thanks for pulling things in. I'm going to keep tuning this; feels like we're on to something here :)

The next issue I'm running into is about ssh keys and git. When a new project is created, app.js logs this error:

[2012-06-01T22:02:05.231Z]  INFO: nodester/13267 on chaos2: Nodester app started on port 4001
[2012-06-01T22:02:41.115Z]  INFO: nodester/13267 on chaos2: gitsetup cmd: sudo /node/nodester/nodester/scripts/gitreposetup.sh /node/nodester/nodester /git testuser 5-fd81e8c266aa05dcc20133f18c0f1af4 hello.js nodester git /app
[2012-06-01T22:02:41.137Z]  INFO: nodester/13267 on chaos2: Update called on ptable /node/nodester/nodester/var/proxy_table.json
[2012-06-01T22:02:41.150Z] ERROR: nodester/13267 on chaos2:
    gitsetup error: Error: Command failed: sudo: no tty present and no askpass program specified
    Sorry, try again.
    sudo: no tty present and no askpass program specified
    Sorry, try again.
    sudo: no tty present and no askpass program specified
    Sorry, try again.
    sudo: 3 incorrect password attempts

looks like it can't sudo for some part of the git repository creation (and fails before even creating any of the directory structure in /app). However, if I manually run the script ./scripts/gitreposetup.sh as root with the same arguments, it works fine.

The second issue I'm having with git is around cloning. It keeps asking me for a password when I try to clone. I'm guessing this is related to RSA keys, but I have no idea. What password should be used when cloning the project for the first time, or should the RSA key take care of that and never ask you for a password?

cheers,
Adam

Edit: maybe the issued is with populating the "SSH KEY" section of env_creation.sh. Which key should go there? and how critical it is to supply that during the install? --thanks!

akumpf · 2012-06-02T01:31:38Z

ok, so quick update.

The issue seems to be that /node/nodester/.ssh/authorized_keys is not being updated. My first guess is permissions, but I'm not familiar enough with the underlying architecture to do the right thing here.

I created one user as a test (don't see anything in authorized_keys), and still don't see any updates to authorized_keys even if I send rsakey updates via curl for the user.

once I manually populated my key (and added ./scripts/gitreposetup.sh to nodesters in the sudoers list; see other pull request), it seems to be setting up the git repo and allowing access as intended :)

Thoughts?

akumpf · 2012-06-03T05:31:57Z

@AlejandroMG am I correct to assume that the nodester user should own both the /app and /git directories (and all of their subdirectories with user apps)?

It looks like after the install root owns those two directories; this causes a fail when pushing apps since /node/nodester/nodester/lib/app.js doesn't have permission to mkdir in a directory owned by root.

roskoff · 2012-06-05T05:24:57Z

@akumpf, your script it's great! I've been testing it (not extensively) and works pretty well by now. I was searching some workaround for the couchdb admin issue, but no luck yet. I think it will be very usefull if you post the script as a gist or even better do PR on nodester-installer ;)

alejandro · 2012-06-05T05:43:34Z

@roskoff what couchdb error?

@akumpf sorry I didn't get the notifications :S As far as I know, git shoud be owned by the user git and root. If I'm not wrong I think we have groups. But git needs to be owned by the user who trigger the post-receive hook. And app are owned by nodester user.

roskoff · 2012-06-05T06:14:32Z

@AlejandroMG, this one:
(#8 (comment))
"1. after the nodester user is added to the database (appending the line nodester = password to /etc/couchdb/local.ini), a full system reboot is need to get the user to be authorized for login -- i'm sure there's a simple fix here somewhere?"

It seems that /etc/init.d/couchdb (start|stop) doesn't reload correctly the configuration settings. I thought it was an issue resolved on a later version and tried to install from git but still have the same problem.

akumpf · 2012-06-06T03:11:01Z

@AlejandroMG and @roskoff looks like the couchdb restart bug has been discussed (albeit a couple years ago here):
https://bugs.launchpad.net/ubuntu/+source/couchdb/+bug/448682

added a fix to the script to force kill couchdb with signal 9:
ps -U couchdb -o pid= | xargs kill -9

after a handful of revs, it's getting border-line useful :) will send a pull request in the next couple hours with everything ready for the next round of issues and discussion.

Thanks to both of you for being excited about this. I think the install process is probably the biggest barrier to others setting up their own Nodester clouds. Hopefully this will open some new doors.

Still some post-install funkiness (permissions, chroot, admin app(s), etc.). If you guys have a minute to try it out (after I send the pull request shortly) and see where it breaks for you, that'd be awesome.

nodester · 2012-06-06T05:28:47Z

We have zipped up our sandbox directory to distribute. You can download it from http://nodester.com/sandbox.tar.gz or find it in the repo under the public directory. You will need to unzip this directory and update your config.js node_base_folder: '', to reference this unzipped directory. New apps start with this sandbox as the base chroot directory.

akumpf · 2012-06-06T06:38:07Z

awesome. Thanks for posting the sandbox.

ok, so everything seems to install nicely, but the permissions are still tripping me up. sudo commands don't behave nicely when they are called from git operations (launching repo apps) as well as when new directories are created (creating a new project). I've played around with removing some of the sudo commands and it does help with some errors, but it seems to introduce others.

@nodester @chrismatthieu @AlejandroMG @roskoff : not sure how many of you have been trying out the install script and have deep knowledge of permissions in nodester, but any thoughts or shell script suggestions would be much appreciated. :)

Oh, to automate the sandbox install, I've added this to the auto-install script:

cd /opt
wget http://nodester.com/sandbox.tar.gz
tar -zxvf sandbox.tar.gz 
rm sandbox.tar.gz

roskoff · 2012-06-06T07:36:34Z

@akumpf You are doing a great work :) Until now I just tested partially the script, I was stuck with the couchdb issue, I asked for help on #couchdb and they recommended to do:

if [ `grep -Fc "nodester" /etc/couchdb/local.ini` != "0" ]
then
    # code if found
    echo "No need to update /etc/couchdb/local.ini, Looks like nodester change was already made."
else
    # code if not found
    curl -X PUT http://127.0.0.1:5984/_config/admins/nodester -d '"password"';
    sleep 1;

fi

And that worked very well, no need to restart the couchdb server.

Now I'm trying rerun the script up to nodejs installation, but I had another error:

# n stable
######################################################################## 100.0%
rm: cannot remove `node-v0.6.19.tar.gz': No such file or directory
Error: installation failed
  node version 0.6.19 does not exist,
  n failed to fetch the tarball,
  or tar failed. Try a different
  version or view /tmp/n.log to view
  error details.

And cat /tmp/n.log says:

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

So now I'm running the script install with sudo n 0.6.18 to keep testing it

akumpf · 2012-06-06T15:37:14Z

cool. ran into the n stable issue last night as well. Looks like 0.6.19 is a release candidate and so the file naming is messed up. I left an issue on @visionmedia/n (since stable should also grab versions that are even numbers anyway).

The CouchDB password fix looks great. Good call.

Have you had any issues with permissions once the installer finishes? Try to create a test project and commit it (keep an eye on /node/logs/app.log).

alejandro · 2012-06-06T15:40:51Z

@akumpf Sorry If i'm missing this conversation commenting, but I'm reading ;-). I'll setup a fresh server tonight with your script, and I'll tell you how things went.

plus I'm not receiving the notifications about this thread.

roskoff · 2012-06-07T13:21:23Z

@akumpf I was able to run the entire script, there is a few things that I should point out:

redis-server is not included by default on Ubuntu 12.04 and the installer scripts doesn't mention it anywhere, so in Installing apt-get packages should go:

sudo apt-get -y install git-core curl build-essential openssl libssl-dev psmisc couchdb htop redis-client

That should install redis 2.2.12 (latest version is 2.4.14)

In INSTALLING NODESTER section, after environment creation we need to change the nodester home dir ownership:

sudo chown -R nodester:nodester /node

(maybe it would be better to set already this on env_creation.sh)

Do we really need to run three times npm update on ENSURE NPM DEPENDENCIES ARE MET section?

The section NodesterHelper: Moving to /node shouldn't be needed, since env_creation.sh already creates the environment on /node, the exception here is that /node/logs doesn't get created. This should be added on env_creation.sh

I think that's all that I get until now, I've tried to set the installer script so it can run as a normal user, using sudo inside it wherever its required.

After I got all this fixed, finally the nodester box was running, but I couldn't get to work it correctly, accessing to the server replies 404 page to every request that I made. I got errors like this on /node/logs/proxy.log:

[2012-06-07T07:49:05.912Z]  INFO: proxy::nodester/15285 on nodestervm: 192.168.0.108:undefined

That's all the tests that I made so far, I know you maybe already have fixed some of then :)

alejandro · 2012-06-07T14:17:51Z

FWIW yesterday I installed a nodester instance with @akumpf's script. It works really well. One thing to take note, since the script is downloading the sandbox.tar.gz directly in the process, we can assume in the config.js script the path for this dir (which is /opt/node-v0.4.9) also it'll be good if we add a more verbose help, before you edit the config file, since it can be really confuse in what and why you need to change that stuff, imho. Besides that, the script really works!! :P

Also about the redis dependency, imo, it should be an optional Dependency, even the couchdb installation, because (at least me, and nodester.com), I don't want to have all in the same server. What are your thoughts about this?

About @roskoff question, it's not an error. That's because you need to add your default IP or Domain to be the principal, as tld_domain in config.js if you put that IP in that space (equally to:

tl_dom:   'testnodester.com',
api_dom:  'api.testnodester.com',
git_dom:  'testnodester.com',

in the config file.) So add that IP or a domain and it will work. Other way to fix this is to add it by default in the proxy table, but since it can be different maybe @akumpf can add it to the script :P.

Edit: Abuot db dependencies, as you can see many people will choose not to use redis _> fgnass/nodester@1c6703a

akumpf · 2012-06-07T23:59:44Z

cool! I'm all for Redis being optional. :)

I think the revisions are starting to get a bit scattered... @AlejandroMG , do you want to pull in everything so far, and then we can pull over the optional Redis config and future updates on top of that?

alejandro · 2012-06-08T03:21:18Z

@akumpf cherry-pick'ed. Please git pull.

alejandro · 2012-06-09T05:43:09Z

I was wondering about the views from couchdb. I need to make a list of them and maybe just after that add them to this script.

roskoff · 2012-06-10T05:00:38Z

Hey guys, do you have trouble installing default npm for node v-0.4.12? Because I do. When nodester_auto.sh checks the dependencies doesn't find npm and when asks if you want it to install all the modules for you it couldn't, because npm is missing.

I've installed it manually: curl http://npmjs.org/install.sh | clean=no sh (with 0.4.12 as current version) and I got npm 1.0.106.

Ok, after that I keep trying to get things to work but then I had problems with global module cradle at the time of running start proxy. To sort it out I had to run manually the comands on the upstart scripts :(

I don't know if those errors are derived from the workaround that I made with npm

@akumpf I've got several error messages when create/init/delete applications, but as I said before, don't know if they are derived issues given a bad installation. Here are some of the error that I've found so far:

When creating:

[2012-06-09T16:31:17.693Z] ERROR: nodester/23947 on nodestervm:
    gitsetup stderr: warning: You appear to have cloned an empty repository.

At init:

[2012-06-09T18:20:40.555Z]  INFO: nodester/23947 on nodestervm: Checking: /app/eocampos/1-d2501fdc1d65d56d235c6ad9692789a9
[2012-06-09T18:20:40.569Z]  INFO: nodester/23947 on nodestervm: Checking: /app/eocampos/1-d2501fdc1d65d56d235c6ad9692789a9/server.js
[2012-06-09T18:20:40.609Z]  WARN: nodester/23947 on nodestervm: App start file does not exist: /app/eocampos/1-d2501fdc1d65d56d235c6ad9692789a9/server.js

When deleting (repeated several times):

[2012-06-09T20:18:44.996Z]  INFO: nodester/24231 on nodestervm:
    sudo umount /app/eocampos/1-d2501fdc1d65d56d235c6ad9692789a9_chroot/dev stderr: Sorry, try again.
    Sorry, try again.
    Sorry, try again.
    sudo: 3 incorrect password attempts

Also I have to say that when I create an app asks me for nodester password (the local user on the server), why could be that?

alejandro · 2012-06-10T22:44:06Z

Well, in my case I didn't fine that problem with npm. I'd recomend to use as default version the 0.4.9 (for the moment), but 0.4.12 shoud be fine. The first two errors that you say are not really errors are warnings. The first is ok, always happen. The second means that your start script on "start process" doesn't exists, do you remember:

> nodester app create APPNAME INITFILE.js

Well that INITFILE.js doesn't exists in your repo.

For the first error, looks like more a permission issue. Try to add the current user to the chown list of that dir.

/be

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nodester simplified auto-installer needed #8

Nodester simplified auto-installer needed #8

akumpf commented May 31, 2012

chrismatthieu commented May 31, 2012

alejandro commented Jun 1, 2012

akumpf commented Jun 1, 2012

akumpf commented Jun 1, 2012

akumpf commented Jun 1, 2012

alejandro commented Jun 1, 2012

akumpf commented Jun 1, 2012

akumpf commented Jun 2, 2012

akumpf commented Jun 3, 2012

roskoff commented Jun 5, 2012

alejandro commented Jun 5, 2012

roskoff commented Jun 5, 2012

akumpf commented Jun 6, 2012

nodester commented Jun 6, 2012

akumpf commented Jun 6, 2012

roskoff commented Jun 6, 2012

akumpf commented Jun 6, 2012

alejandro commented Jun 6, 2012

roskoff commented Jun 7, 2012

alejandro commented Jun 7, 2012

akumpf commented Jun 7, 2012

alejandro commented Jun 8, 2012

alejandro commented Jun 9, 2012

roskoff commented Jun 10, 2012

alejandro commented Jun 10, 2012

Nodester simplified auto-installer needed #8

Nodester simplified auto-installer needed #8

Comments

akumpf commented May 31, 2012

chrismatthieu commented May 31, 2012

alejandro commented Jun 1, 2012

akumpf commented Jun 1, 2012

akumpf commented Jun 1, 2012

akumpf commented Jun 1, 2012

alejandro commented Jun 1, 2012

akumpf commented Jun 1, 2012

akumpf commented Jun 2, 2012

akumpf commented Jun 3, 2012

roskoff commented Jun 5, 2012

alejandro commented Jun 5, 2012

roskoff commented Jun 5, 2012

akumpf commented Jun 6, 2012

nodester commented Jun 6, 2012

akumpf commented Jun 6, 2012

roskoff commented Jun 6, 2012

akumpf commented Jun 6, 2012

alejandro commented Jun 6, 2012

roskoff commented Jun 7, 2012

alejandro commented Jun 7, 2012

akumpf commented Jun 7, 2012

alejandro commented Jun 8, 2012

alejandro commented Jun 9, 2012

roskoff commented Jun 10, 2012

alejandro commented Jun 10, 2012