Latest Entries »

Server restart manual

AFTER RESTART:
1. restart nginx
2. restart postgresql


/usr/local/bin/postmaster -D /usr/local/postgresql/data > postgresql.log 2>&1 &

3. restart rabbitmq


/etc/init.d/rabbitmq-server start

4. restart postfix/dovecot
5. open iptables
6. restart django


$ ssh -i XXX.pem ec2-user@xxx.xxx.xxx.xxx

[ec2-user@ip-xxx-xxx-xxx-xxx ~]$ cat <<EOF > python_install.sh
  cd /root
  mkdir Downloads
  cd Downloads/
  wget https://www.python.org/ftp/python/3.4.1/Python-3.4.1.tgz
  wget https://www.python.org/ftp/python/2.7.8/Python-2.7.8.tgz
  tar -xzvf Python-3.4.1.tgz 
  tar -xzvf Python-2.7.8.tgz 
  yum install -y gcc ncurses-devel ncurses-libs ncurses sqlite-devel readline-devel openssl-devel xz-devel bzip2-devel
  cd Python-3.4.1
  ./configure --prefix=/usr/local/python-3.4.1
  make
  make install
  cd Python-2.7.8
  ./configure --prefix=/usr/local/python-2.7.8
  make
  make install
  echo 'export PATH=/usr/local/python-3.4.1/bin:/usr/local/python-2.7.8/bin:$PATH' >> /home/ec2-user/.bashrc
EOF

[ec2-user@ip-xxx-xxx-xxx-xxx ~]$ sudo su root python_install.sh
[ec2-user@ip-xxx-xxx-xxx-xxx ~]$ source ~/.bashrc

Django Tips & Hacks

take care of __unicode__ for model

take care of reverse_lazy

Using template to start app: https://github.com/fortable1999/dj-app

Install using yum

yum install postfix dovecot libgsasl-devel

Edit Dovecot configure

1. config dovecot.conf

vim /etc/dovecot/dovecot.conf


add/uncomment these

protocols = imap pop3
mail_location = maildir:~/Maildir
listen = *

2. config 10-mater.conf

vim /etc/dovecot/conf.d/10-master.conf


add these lines into service auth {…}

 unix_listener /var/spool/postfix/private/auth {
 mode = 0666
 user = postfix
 group = postfix
}

Config Postfix

vim /etc/postfix/main.cf

Add these lines:

mydomain = XXX.com
myorigin = $mydomain
inet_interfaces = $myhostname, localhost
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
# mynetworks_style = host
virtual_mailbox_domains = hash:/etc/postfix/virtual
relay_domains = gmail.com, google.com, xxx.com

virtual_alias_maps = hash:/etc/postfix/virtual
home_mailbox = Maildir/

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
# smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks, check_relay_domains
smtpd_recipient_restrictions =
        permit_sasl_authenticated,
        reject_unauth_destination,
broken_sasl_auth_clients = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

restart server

/etc/init.d/dovecot restart
/etc/init.d/postfix restart
/etc/init.d/saslauthd restart

nginx


# nginx installation
sudo useradd nginx
sudo yum instal pcre-devel
cd ~/Downloads
wget http://nginx.org/download/nginx-1.5.12.tar.gz
tar -xzvf nginx-1.5.12.tar.gz
cd nginx-1.5.12
./configure --prefix=/usr/local/nginx
make
sudo make install
sudo chown -R nginx:nginx /usr/local/nginx

postgres


# postgresql installation
cd ~/Downloads
wget http://ftp.postgresql.org/pub/source/v9.3.4/postgresql-9.3.4.tar.gz
tar -xzvf postgresql-9.3.4.tar.gz
cd postgresql-9.3.4
./configure --prefix=/usr/local/postgresql
make
make install
sudo createuser postgres
sudo passwd postgres
# initial database
mkdir /usr/local/postgresql/data
chown postgres:postgres -R /usr/local/postgresql/data
su - postgres
/usr/local/postgresql/bin/initdb -D /usr/local/postgresql/data/
# start postgresql
/usr/local/postgresql/bin/postmaster -D /usr/local/postgresql/data/ > postgresql.log 2>&1 &
cat postgresql.log
# create database
/usr/local/postgresql/bin/createdb testdb
/usr/local/postgresql/bin/psql testdb

elasticsearch

cd ~/Downloads/
wget http://download.oracle.com/otn-pub/java/jdk/7u51-b13/jdk-7u51-linux-x64.tar.gz
tar -xzvf jdk-7u51-linux-x64.tar.gz
sudo mv jdk1.7.0_51 /usr/local/.
sudo cp -si /usr/local/jdk1.7.0_51/bin/* /usr/local/bin/
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.1.0.tar.gz
tar -xzvf elasticsearch-1.1.0.tar.gz
sudo mv elasticsearch-1.1.0 /usr/local/.
sudo useradd elasticsearch
sudo chown -R elasticsearch:elasticsearch /usr/local/elasticsearch-1.1.0



memcahced installation


# memcached installation
sudo yum install libevent-devel
cd ~/Downloads/
wget http://www.memcached.org/files/memcached-1.4.17.tar.gz
tar -xzvf memcached-1.4.17.tar.gz 
cd memcached-1.4.17
./configure --prefix=/usr/local/memcached
make
sudo make install
# start memcached
/usr/local/memcached/bin/memcached &

test


# install Python 3.3.5
sudo yum install ncurses-devel ncurses-libs ncurses sqlite-devel readline-devel openssl-devel xz-devel bzip2-devel
mkdir ~/Downloads
cd ~/Downloads
wget wget http://www.python.org/ftp/python/3.3.5/Python-3.3.5.tgz
tar -xvf Python-3.3.5.tgz
cd Python-3.3.5
./configure --prefix=/usr/local/python-3.3.5
make
sudo make install
cp -s /usr/local/python-3.3.5/ /usr/local/bin
cp -s /usr/local/python-3.3.5/ /usr/bin

# create virutalenv for mangaconnect
cd ~
pyvenv-3.3 production_env
cd production_env
source bin/activate
wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | python
curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
python get-pip.py

Zhao’s Git cheat sheet

—–Not completed————-

Difference comparing

  • compare two different branch:
    
     git diff dev master
     
  • compare one file in different branch:
    
     git diff dev master -- hello.py
     

Rollback

  • undo one commit:
    
    git reset HEAD^ --soft
    
  • undo 3 commit:
    
    git reset HEAD^3 --soft
    

Overview
What is BDD? wikipedia
The procedure of BDD should be:
Producure write features file like:
Programmer write code to fill all requirements in the features
Testing. if all features are implemented properly then release. Or back to 2.
BDD using Cucumber, Capybara, PhantomJS
This material is based on this sample project:
git clone https://github.com/fortable1999/BDD-demo.git
Requirement of this tutorial:
We will work on
Sinatra
Cucumber
Capybara
Node.js
Phantom.js
SimpleCov
Make sure Node.js and Phantom.js can be find in your $PATH
Cucumber
Cucumber is a BDD framework.
There are 4 modules in our Sinatra project:
gem ‘sinatra’
gem ‘haml’

gem ‘cucumber’
gem ‘capybara’
gem ‘poltergeist’ # Also, make sure PhantoJS is installed
gem ‘rspec-expectations’

gem ‘simplecov’
Then, use bundle or gem to install:
bundle install

Writing a feature file in features/story1.feature:
#encoding: utf-8

Feature: Viewer vists the page
Test 4 links redirection

Scenario: goto home from home page
Given I am on the home page
When I click “home” link
Then I goto ‘home’ page

Scenario: goto form from page
Given I am on the home page
When I click “form” link
Then I goto ‘form’ page

Scenario: goto success from home page
Given I am on the home page
When I click “success” link
Then I goto ‘success’ page

Scenario: goto error from home page
Given I am on the home page
When I click “error” link
Then I goto ‘error’ page

Generate step function with cucumber:
$ cucumber
#encoding: utf-8
Feature: Viewer vists the page
Test 4 links redirection

Scenario: goto home from home page # features/story1.feature:6
Given I am on the home page # features/story1.feature:7
When I click “home” link # features/story1.feature:8
Then I goto ‘home’ page # features/story1.feature:9

Scenario: goto form from page # features/story1.feature:11
Given I am on the home page # features/story1.feature:12
When I click “form” link # features/story1.feature:13
Then I goto ‘form’ page # features/story1.feature:14

Scenario: goto success from home page # features/story1.feature:16
Given I am on the home page # features/story1.feature:17
When I click “success” link # features/story1.feature:18
Then I goto ‘success’ page # features/story1.feature:19

Scenario: goto error from home page # features/story1.feature:21
Given I am on the home page # features/story1.feature:22
When I click “error” link # features/story1.feature:23
Then I goto ‘error’ page # features/story1.feature:24
4 scenarios (4 undefined)
12 steps (12 undefined)
0m0.022s

You can implement step definitions for undefined steps with these snippets:

Given(/^I am on the home page$/) do
pending # express the regexp above with the code you wish you had
end

When(/^I click “(.*?)” link$/) do |arg1|
pending # express the regexp above with the code you wish you had
end

Then(/^I goto ‘home’ page$/) do
pending # express the regexp above with the code you wish you had
end

Then(/^I goto ‘form’ page$/) do
pending # express the regexp above with the code you wish you had
end

Then(/^I goto ‘success’ page$/) do
pending # express the regexp above with the code you wish you had
end

Then(/^I goto ‘error’ page$/) do
pending # express the regexp above with the code you wish you had
end

Copy these generated code to a step file: features/step_definition/step1.rb,
and modify pending lines like this:
Given(/^I am on the home page$/) do
visit ‘/’
page.should have_content ‘HelloWorld’
end

When(/^I click “(.*?)” link$/) do |arg1|
named_link = arg1
click_link named_link
end

Then(/^I goto ‘form’ page$/) do
current_path.should == ‘/form’
end

Then(/^I goto ‘success’ page$/) do
current_path.should == ‘/success’
end

Then(/^I goto ‘error’ page$/) do
current_path.should == ‘/error’
end

Then(/^I goto ‘home’ page$/) do
current_path.should == ‘/’
end

Run cucumber, get test result:
#encoding: utf-8
Feature: Viewer vists the page
Test 4 links redirection

Scenario: goto home from home page # features/story1.feature:6
Given I am on the home page # features/step_definitions/step1.rb:1
When I click “home” link # features/step_definitions/step1.rb:6
Then I goto ‘home’ page # features/step_definitions/step1.rb:23

Scenario: goto form from page # features/story1.feature:11
Given I am on the home page # features/step_definitions/step1.rb:1
When I click “form” link # features/step_definitions/step1.rb:6
Then I goto ‘form’ page # features/step_definitions/step1.rb:11

Scenario: goto success from home page # features/story1.feature:16
Given I am on the home page # features/step_definitions/step1.rb:1
When I click “success” link # features/step_definitions/step1.rb:6
Then I goto ‘success’ page # features/step_definitions/step1.rb:15

Scenario: goto error from home page # features/story1.feature:21
Given I am on the home page # features/step_definitions/step1.rb:1
When I click “error” link # features/step_definitions/step1.rb:6
Then I goto ‘error’ page # features/step_definitions/step1.rb:19

4 scenarios (4 passed)
12 steps (12 passed)
0m0.825s
Coverage report generated for Cucumber Features to /home/fortable1999/Workspace/RubyEnv/test2/BDD-demo/coverage. 26 / 26 LOC (100.0%) covered.

You can get coverage from ./coverage:
Rakuten Chef Project > Behavior-Driven Development with Cucumber, Capybara, PhantomJS > code_coverage.png

How to quickly start a django project in rails way

Situation:

You want to create a scaffold,
Project’s name is “another”, with a application ‘blogs’ in it,
‘blogs’ application has a model named Post, which has
a string field named ‘title’,
a text field named ‘body’,
a boolean field named ‘is_public’.
You also want a api binded with the Post model.
The application should have full CRUD function(Create, Remove, Update, Detail)

Preparing:

requirement.txt:

Django==1.5.1
Markdown==2.3.1
PIL==1.1.7
PyYAML==3.10
South==0.8.1
defusedxml==0.4.1
django-filter==0.6
-e git://github.com/fortable1999/django-generate-scaffold.git#egg=django_generate_scaffold
django-oauth-plus==2.1.2
django-oauth2-provider==0.2.6
djangorestframework==2.3.6
gunicorn==17.5
httplib2==0.8
oauth2==1.5.211
shortuuid==0.3
wsgiref==0.1.2

save above lines into a requirement.txt file,
then:

pip install -r requirement.txt

Howto:

1. Start a Django project from zero:

django-admin.py startproject another
cd another/

2. Edit setting.py:

vim another/settings.py

Fix here:

'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'dev.db',

uncomment this:

'django.contrib.admin',

add these lines into APPS_INSTALLED:

'generate_scaffold',
'rest_framework',
'south',
'blogs',

add there lines to setting.py:

REST_FRAMEWORK = {
    # Use hyperlinked styles by default.
    # Only used if the `serializer_class` attribute is not set on a view.
    'DEFAULT_MODEL_SERIALIZER_CLASS':
    'rest_framework.serializers.HyperlinkedModelSerializer',
    # Use Django's standard `django.contrib.auth` permissions,
    # or allow read-only access for unauthenticated users.
        'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.DjangoModelPermissionsOrAnonReadOnly'
    ]
}

3. Create a empty application:

django-admin.py startapp blogs

6. open a url pattern to application:

vim another/urls.py

uncomment these lines:

from django.contrib import admin
admin.autodiscover()
url(r'^admin/', include(admin.site.urls)),

add there new url pattern:

url(r'^', include('blogs.urls')),

4. create scaffold:

python manage.py generatescaffold blogs Post title:string body:text is_public:bool

5. edit application’s sub urls.py:

vim blogs/urls.py

add there lines after import part:

from django.conf.urls import include
from rest_framework import viewsets, routers
from models import Post
class PostViewSet(viewsets.ModelViewSet): 
    model = Post 
router = routers.DefaultRouter() 
router.register(r'posts', PostViewSet)

then add this into urlpatters:

url(r'api/', include(router.urls)),

6. prepare development envionment database:

python manage.py syncdb
python manage.py schemamigration --initial blogs
python manage.py migrate blogs

7. testing project:

python manage.py runserver

Project done!

References:

Django Rest Framework: http://django-rest-framework.org/
Django generate scaffold: https://github.com/fortable1999/django-generate-scaffold

test

test

Follow

Get every new post delivered to your Inbox.