Macaw Overview

Macaw is an enterprise grade prescriptive Microservices platform. Macaw provides comprehensive toolset, many built-in core services, CI/CD integrations, management & operational capabilities and cloud agnostic deployment to accelerate enterprise cloud native journey.

Macaw overview:

  • API modeling in JSON or Yang and Swagger documentation for microservices, or collection of microservices called as blueprints.
  • Codegen, build and publish toolset for Microservice development. Java, Javascript language for service implementation, with more languages support coming in future.
  • Many built-in core, essential services like database (Cassandra, MySQL), logging (Elasticsearch), identity with multi-tenancy etc. for rapid app development and focus on business logic.
  • Microservice metadata allows for standardization, auditing, policy enforcement and service discovery & provisioning. Microservice tag identifies a specific code commit and can be used for release/sprint progression, staging or production usage.
  • Macaw microservices are packaged as Docker containers. MDR (Meta Data Repository) holds the service blueprints, Metadata information, available Docker tags for a specific service. Docker registry holds service container images.
  • Macaw service catalog serves as a marketplace and presents microservices that are available in selected MDR/Docker repositories, which can be private, shared or public.
  • Macaw provides macaw CLI and a graphical web based console to deploy, scale and administer (monitor, troubleshoot) platform and the microservices. Message co-relation and event visualization serve as essential troubleshooting and analytics tools.
  • From a runtime standpoint, Macaw is a distributed app running on top of one or more standalone Docker nodes that host Macaw microservices. Messaging, routing and loadbalancing among the nodes and services is achieved using Kafka & Zookeeper over SSL and frontend UI messages are loadbalanced through HAProxy. For service level resiliency, Macaw creates multiple instances of a microservice and operates it as a cluster using its native placement, load balancing algorithm.
  • Macaw can be configured to send container placement request to a resource scheduler like Kubernetes or Docker Swarm, which will assume responsibility of container placement and high availability.
  • Macaw offers several installation choices. For on-prem, Macaw can be installed on developer machine (Mac/PC/Linux) using Vagrant and VirtualBox, or on VMware vSphere. For clouds, Macaw AMIs are available on AWS Cloud and for other clouds, Macaw can be installed on top of CentOS based Linux nodes using installer scripts.

Setup and Installation

Following sections will cover Macaw platform installation and development environment setup on a MAC(OSX). For information on other installation options refer to documentation.

In the below steps, you are going to launch a CentOS Virtual machine using Vagrant/Virtualbox and run macaw platform on it. On the local MAC system, you will follow the instructions to install necessary tools tools like Docker and MACAW SDK to be able to develop micro services, publish and provision them on the macaw platform. From here onwards, Local Host/System will be referring to your MAC System and platform/vagrant/macaw VM refers to the Virtual Machine which is launched using Vagrant/Virtualbox

Before proceeding further, please go through the download instructions.

Download

Request Macaw Download by providing your contact information. You will receive a download link with all the details in an E-mail after accepting EULA.

Requirements

System:
  • MAC System Running OS X
  • Memory - Minimum 12GB
  • CPU - Minimum 2 Cores
Software/Tools:

Below is a list of software requirements needed to bring up macaw platform and also the necessary development environment to be able to develop/publish and deploy microservices. If you already have working versions of any of these tools just make sure to verify the versions and skip the installation. If not present, follow the link for each and do the required installation as suggested.

  • Vagrant - Version 1.8.7 (or above)
  • Virtual Box - Version 5.1.x (or above)
  • Note: Click here to verify your vagrant/virtual box versions and sanity of the installation.

  • Oracle JDK - Version 8 (or above) (Skip this step if java is already present)
  •  
    java -version
    
  • Apache ant - Version 1.9.x (or above) with Ant-contrib.
  •  
    curl -o /tmp/ant-linux.tar.gz  -k http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.7-bin.tar.gz
    sudo mkdir -p /opt/ant/
    sudo tar xf /tmp/ant-linux.tar.gz -C /opt/ant --strip-components 1
    rm -rf /tmp/ant-linux.tar.gz
    
    vi ~/.bashrc and add the below lines.
    
    ANT_HOME=/opt/ant
    export ANT_HOME
    export PATH=$ANT_HOME/bin:$PATH
    
    source ~/.bashrc
    
  • Python 2.7.x and dependent packages. Python is installed by default on most of the systems. Check the version and skip python installation if version is 2.7.x
  •  
    # python --version 
    

    Install the necessary python modules using pip command.

    # curl -fsSL https://bootstrap.pypa.io/get-pip.py | sudo python 
    # sudo pip install requests jsonschema 
    
  • Docker 1.12.x (or above) (Use Stable Channel)
  • Note: Make sure to follow all 3 three steps including the Docker Installation verification.

Launching the Platform VM

In the below section, you will be launching a Virtual machine using Vagrant/Virtualbox. Macaw platform will be installed on this VM.

VM Launch steps
vagrant box add < Link to the Vagrant Box. You will receive this as part registration confirmation email >
vagrant box list
mkdir ~/Documents/macaw>
cd ~/Documents/macaw
vagrant init macaw
vagrant up
vagrant ssh

Note: The above steps download the box image from macaw.io and the completion time would be dependent on the network speed.

Sanity Check

On completion of above steps you are now logged into Platform VM. Note down the IP address of the VM using the following command. The Virtual Machine is pre-configured with the IP 192.168.33.10. This IP is reachable only from your current host. The VM will be able to access the rest of the internet.

 
[macaw@mplatform ~]$ hostname
mplatform
[macaw@mplatform ~]$ ip addr
enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:78:7e:9f brd ff:ff:ff:ff:ff:ff
inet 192.168.33.10/24 brd 192.168.33.255 scope global enp0s8

In this case the IP is

192.168.33.10

Note: The Vagrant VM can be customized by editing the Vagrantfile before issuing the command vagrant up. Follow documentation from Vagrant on how to edit Vagrantfile.

Platform Installation

Execute the below sequence of steps on Vagrant/Platform VM to complete platform configuration and installation. You will receive the

tag
as part of the downloads registration confirmation email.

Execute macaw setup to generate the platform and provisioner configuration. Accept EULA when prompted. For Platform Host and Service Host, provide the IP 192.168.33.10 that you noted earlier. For all other settings, you can hit enter to select the defaults and type yes to accept.

Setup
[macaw@mplatform ~]$ macaw setup
FQDN of platform Host [localhost.localdomain]: 192.168.33.10
FQDN or services hosts (, separated if more than one): 192.168.33.10

Bootstrap Macaw Infra

 [macaw@mplatform ~]$ macaw infra install --tag <tag> 
Bootstrap Macaw Platform

During the dbinit step in the below commands, user would be prompted to enter Platform Admin Username and Password. By default the user would be 'admin@macaw.io' and the password is 'admin'. You can chose to configure your specific platform admin user with a different password. The username is an email address.

[macaw@mplatform ~]$ macaw platform dbinit --tag <tag> 
[macaw@mplatform ~]$ macaw platform install --tag <tag> 
Bootstrap Macaw onprem Tools
 
[macaw@mplatform ~]$ macaw tools install --tag <tag> --service macaw-mdr 
[macaw@mplatform ~]$ macaw tools install --tag 2.3.1 --service docker-registry
[macaw@mplatform ~]$ sudo mkdir -p /etc/docker/certs.d/192.168.33.10:5000
[macaw@mplatform ~]$ sudo cp /opt/macaw-config/certificates/ca/ca.crt /etc/docker/certs.d/192.168.33.10:5000/ca.crt

Note: The docker registry tag is pulled from the public docker registry.

Verify the Installation
 
[macaw@mplatform ~]$ macaw status 

The above command would display the status of the Macaw Infra/Platform/Tools. If you notice any container exiting/restarting, check the documentation/support team (support@macaw.io) for further troubleshooting.

If no errors were encountered till this point then you have successfully completed platform installation.

Login to Macaw Console

Once the above steps are executed, macaw console can be accessed at:

https://192.168.33.10

Login with username/password specified during setup. Default login is shown below.

Username:

admin@macaw.io

password:

admin

Get familiar with the interface. Navigate to Services tab to view running services. Click on any of the service to view its details, or browse/invoke APIs.



Fig. Macaw Console Login


Fig. Microservice Clusters


Fig. Microservice Details


Fig. Microservice API


SDK and Dev Environment

Once Macaw platform is up and running, you can quickly build and deploy example microservices that are packaged in SDK. To build more comprehensive and real-world Microservice applications refer to SDK developer guide

Download and Setup the SDK

As part of the download registration confirmation email, SDK downloadable link would be sent. Download the macaw SDK and untar it into your local project development directory.

# mkdir -p ~/Documents/workspace
# cd ~/Documents/workspace
# tar xf {Location of the Download SDK Tar}

Add the below entries to your ~/.bashrc

export MACAW_SDK_HOME=~/Documents/workspace/macaw-sdk
export MACAW_SERVICES_HOME=~/Documents/workspace
export PATH=$MACAW_SDK_HOME/tools/macaw-publish-tools/bin:$PATH

Source the environment

source ~/.bashrc

Note: From here onwards, these environment variables will be referred in the entire documentation. Some of the macaw tools also refer to these environment variables to locate the services you are trying to publish.

MACAW_SDK_HOME - Directory where your downloaded macaw SDK artifacts are present.

MACAW_SERVICES_HOME - Directory where you are developing your microservices

MDR and Docker Registry

Copy the

macawpublish.globals
file from the output of the tools installation. The default path is
/opt/macaw-config/macaw-tools
on your platform VM. Copy the file to the SDK directory at the below location
$MACAW_SDK_HOME/tools/macaw-publish-tools

# cd $MACAW_SDK_HOME/tools/macaw-publish-tools/
# sftp macaw@192.168.33.10:/opt/macaw-config/macaw-tools/*.globals .
macaw@192.168.33.10's password: macaw

The onprem Docker Registry is setup with self signed private certiicates. It is necessary to setup up the trust for the docker daemon on Host (MAC Host) to be able to communicate with onprem docker registry installed as part of the tools. Below are the instructions on how to achieve this for Host (MAC). Please refer to  macawpublish documentation for more details.

On the Host (MAC)

Step 1: Download the ca.crt in to your /Users/<id>/Documents folder.

# sftp macaw@192.168.33.10:/opt/macaw-config/certificates/ca/ca.crt ~/Documents/
macaw@192.168.33.10's password: macaw

Step 2: Connect to the Docker VM and Copy Certificate

# screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
# mkdir -p /etc/docker/certs.d/192.168.33.10:5000
# cp /Users/{your user Id on MAC}/Documents/ca.crt /etc/docker/certs.d/192.168.33.10:5000/ca.crt
Type Control + a followed by Control + \ to exit your screen session

Step 3: Verify docker login from the host 

# docker login -u macaw -p macaw@local -e macaw@local.com 192.168.33.10:5000

Once this point is reached successfully, macaw platform and development environment is setup successfully to be able to develop new micro services and be able to publish and provision them.

Note: Due to limitation with docker version 1.12.x on MAC, if your system or docker restarts, you have to repeat these 3 steps. The copied certificate is lost.


Develop Microservice

Once SDK and development environment setup is complete, you are ready to start developing and running microservices in Macaw platform. In this guide, we will pick one of the SDK quickstart examples. To develop new microservices you can clone SDK quickstarts or develop new microservices by referring to SDK Developer guide.

Develop Macaw Microservice

Macaw Microservices currently support Java and Javascript languages. Go to

quickstarts
directory and pick
calculator
microservice.

Model your Microservice.


You can use the existing calculator sample and build on top of it.
# mkdir -p $MACAW_SERVICES_HOME/my_service_yang
# cp $MACAW_SDK_HOME/quickstarts/calculator/api/yang/calculator.yang $MACAW_SERVICES_HOME/my_service_yang/
# cd $MACAW_SERVICES_HOME/my_service_yang/
Modify/add to the yang as per you need. 

module calculator {

    prefix "calc";
    namespace "http://macaw.io/quickstart/service/calculator";
    yang-version 1;

    import macaw-service-extension {
        prefix s;
    }

    organization "macaw.io";
    contact yang-model-info@macaw.io;

    description "A simple calculator service";


    revision 2016-06-08;

    container service-def {
        s:service-namespace "io.macaw.services";
        s:service-name "calculator";
        s:options {
            s:java-options {
                s:service-interface-class-simple-name: "Calculator";
                s:service-impl-class-simple-name: "BasicCalculator";
            }
        }
    }
    
    rpc add {
        description "Adds the numbers given as input";

        input {
            leaf-list numbers {
                type decimal64 {fraction-digits 2;}
                description "Add all numbers";
                
            }
        }

        output {
            leaf result {
                type decimal64 {fraction-digits 2;}              
            }
        }

    }

    rpc multiply {
        description "Multiplies the numbers given as input";

        input {
            leaf-list numbers {
                type decimal64 {fraction-digits 2;}
                description "Multiplies all numbers";
            }
        }

        output {
            leaf result {
                type decimal64 {fraction-digits 2;}              
            }
        }
    }
}


Fig. Yang Model for sample Calculator Macaw Microservice API

Artifact Generation Properties

Once you have modeled all your APIs then, then you specify how and where you would like to generate the artifacts in a simple properties file.

# cd $MACAW_SDK_HOME/tools/macaw-service-artifacts-generator
# cd conf
# vi service-artifacts-gen.properties

Modify the service artifact properties with highlighted values shown below. Leave defaults for rest

# The version of the module/project. Example: 1.0.0
module.version=1.0.0

# The type of the source model from which the code will be generated. Valid values are "yang", "json"
source.model.type=yang

# The full path to the directory containing the yang files from which the artifacts will be generated.
# Example: /home/me/yang/
yang.files.dir=$MACAW_SERVICES_HOME (Provide the Full Path. Environment Variables are not supported here)

# The full path to the directory where the generated artifacts need to be placed
# Example: /Users/foo/myyangs/
artifacts.output.dir=$MACAW_SERVICES_HOME/my_service_yang (Provide the Full Path. Environment Variables are not supported here)

# The build tool that will be used for the generated project.
# Valid values are "ivy" or "maven" or "plain-ant" (without those quotes).
# At this time, we only support "plain-ant".
project.build.tool=plain-ant

# The generated project can be built and deployed/released to release repositories. The organization name
# corresponds to the name of the organization under which this project's artifacts will be released.
# Example: io.macaw.examples
project.dist.organization.name=io.macaw.examples

# The generated project will depend on Macaw service platform/runtime. The dependency version here specifies
# the version of the Macaw platform on which the generated project depends
# Example: 0.9.4
macaw.platform.version=0.9.0-Alpha-3-SNAPSHOT


# whether impl part of service needs to be generated afresh
# false(default) -  will check for impl folder, if not present will generate both api and impl else only api is generated
# true -  will generate/overwrite the api and impl
# Example: true/false
service.gen.option.generate.impl.artifacts=false

Fig. Service Artifact Properties file for Macaw Microservice

Note: Replace the environment variables with absolute paths. The code generation tool doesnt support environment variables yet.


Codegen

Next step is to use our code generator tool from Macaw SDK to generate required artifacts and stubs that are ready for you to go ahead and implement your Microservice. You will have separate stubs for public API implementation and internal service implementation.

cd $MACAW_SDK_HOME/tools/macaw-service-artifacts-generator/bin
./run.sh $MACAW_SDK_HOME/runtime/macaw-service-runtime.tar.gz

Note: At this point your service directory $MACAW_SERVICES_HOME/calculator is created. Go through the detailed documentation to understand the directory structure of the service.

Implement business logic in the Microservice

Provide business logic and implementation in the java/javascript source files generated in impl directory.

# cd $MACAW_SERVICES_HOME/calculator/impl/
# vi src/main/java/...../impl/BasicCalculator.java

Provide implementation for

add
and
multiply
methods similar to the highlighted text shown below

  @Override
  public double add(double[] numbers) {    
    double sum = 0.0d;
    for (final double num : numbers) {
      sum+= num;
    }
    return sum;
  }

  @Override
  public double multiply(double[] numbers) { 
    double result = 0.0d;
    for (int i = 0; i < numbers.length; i++) {
      if (i == 0) {
	result = numbers[i];
	continue;
      }
      result = result * numbers[i];
    }
    return result;
  }

Building the Microservice

Build your Microservice using generated build files with familiar tools like ant. This will generate the target executable jar files.

# cd $MACAW_SERVICES_HOME/calculator/api/
# ant clean deploy
# cd $MACAW_SERVICES_HOME/calculator/impl
# ant clean deploy -Dmacaw.service.runtime.binary=$MACAW_SDK_HOME/runtime/macaw-service-runtime.tar.gz

Now you are ready to containerize this Microservice and publish to repository.

Publish the Microservice

Create and publish the Microservice Blueprint

Macaw blueprint is a collection of related Microservices that function like a module. For example, you can have a blueprint called ‘Infrastructure Services’ and package all related infrastructure Microservices in the blueprint.

In this example, we only have one service in the blueprint.
Note: Creation and publishing of blueprint is a one time process and need not be repeated for the same service.
Create Blueprint:

# macawpublish create calculator 
.. 
Blueprint create successfully and stored at:
....../macaw-sdk/tools/macaw-publish-tools/bin/../service-blueprints/macaw-service-bp-0f837303-a3c5-59ab-ae77-f0fa68b13ca5.json 
.. 
Blueprint can be uploaded to MDR using macawpublish blueprint <Blueprint File> Publish Blueprint:
# macawpublish blueprint <Blueprint File> 
.. 
Blueprint macaw-service-bp-0f837303-a3c5-59ab-ae77-f0fa68b13ca5.json published successfully. 
.. 

Fig. Service Blueprint file for Macaw Microservice

Publish the Microservice

Once we have created and published the blueprint, it’s time to publish the Microservice that we have built.

Publish the Microservice metadata, tag name. Along with the metadata publishing, this will also build container image and publish to docker registry

# macawpublish service --tag macaw-demo calculator

Deploy the Microservice

Login to the Macaw DevOps console


Browse to Catalog and select the onprem repository.



Locate your Microservice


Deploy the Microservice. Make sure to select the correct tag you used when you published the Microservice. In a production environment, this also means release team can pickup code from your SCM like SVN, GitHub, BitBucket etc. and apply tags to the services. QA team can pickup these tags and perform validations and announce sanitized versions through tags.



Once you have deployed your service, you can locate the running version of your instance in Services tab






Now that your service is up and running, you can browse the API, invoke APIs from API browser.

Videos

 

 
Ready to Macaw?
Try for free
© Copyright 2015 - 2021 Macaw Software Inc. All Rights Reserved.