DEV Community

Cover image for Create AWS Lambda + Apigateway + Opensearch
Marcos Giordano
Marcos Giordano

Posted on

Create AWS Lambda + Apigateway + Opensearch

Create OpenSearch Domain:

  • On OpenSearch Dashboard -> Create Domain
  • Domain name : mydomain -> could be any (just remember it)
  • Deployment type : Development and Testing
  • Data nodes :
    • Instance type : t3.small.search
  • Network : Public access
  • Fine-grained access control :
    • Enable fine-grained access control : should be enabled
    • Create master user :
      • Set Master username
      • Set Master password
  • Access policy: Only use fine-grained access control
  • We will continue configurating the system, as openSearch takes some time to effectively be created...

Image description

Create Lambda Function

  • On Lambda Dashboard -> Create function
  • Create Function : Author from scratch
  • Set Function Name -> open-search-lambda-function -> could be any (just remember it)
  • Create Function
  • On the newly created function, go to code and in the index.js file replace the code to this:
const AWS = require('aws-sdk');

var region = 'us-east-1'; //region where openSearch has been deployed
var domain = process.env.DOMAIN_URL; // environment variable that we have to set
var index = 'movies'; //table name in openSearch
var api_type = '_search'; //search method in openSearch

const searchDocument = async (key) => {
    try {

        let query = {
            "size": 25,
            "query": {
                "multi_match": {
                    "query": key,
                    "fields": ["Title^4", "Plot^2", "Cast"]
                }
            }
        };
        //creating a request body
        var endpoint = new AWS.Endpoint(domain); //creating Endpoint
        var request = new AWS.HttpRequest(endpoint, region); //creating request body with endpoint and region
        request.method = 'GET';  // method PUT, POST, GET & Delete
        request.path += index + '/' + api_type + '/';
        request.body = JSON.stringify(query);
        request.headers['host'] = domain;
        request.headers['Content-Type'] = 'application/json';
        request.headers['Content-Length'] = Buffer.byteLength(request.body);

        console.log('OpenSearch Request: ', { request });


        //Signing the request with authorized credentails like IAM user or role
        var credentials = new AWS.EnvironmentCredentials('AWS');
        var signer = new AWS.Signers.V4(request, 'es');
        signer.addAuthorization(credentials, new Date());

        //http request to the server
        var client = new AWS.HttpClient();
        return new Promise((resolve, reject) => {
            client.handleRequest(
                request,
                null,
                function (response) {
                    console.log(response.statusCode + ' ' + response.statusMessage);
                    var responseBody = '';
                    response.on('data', function (chunk) {
                        responseBody += chunk;
                    });
                    response.on('end', function (chunk) {
                        console.log('Response body: ' + responseBody);
                        resolve(responseBody);
                    });
                },
                function (error) {
                    console.log('Error: ' + error);
                    reject(error);
                }
            );
        });
    } catch (err) {
        console.log(err);
    }
};

exports.handler = async (event) => {

    console.log("Event: ", JSON.stringify(event));

    const key = event.queryStringParameters.key;
    console.log("key", key);

    //calling the function to query
    let res = await searchDocument(key);
    console.log("Results Fetched..........");
    res = JSON.parse(res);

    //remove additional data
    let records = [];
    res.hits.hits.forEach(data => {
        records.push(data._source);
    });

    // TODO implement
    const response = {
        statusCode: 200,
        body: JSON.stringify(records)
    };
    return response;
};
Enter fullscreen mode Exit fullscreen mode
  • Once you pasted the code in the lambda function -> file -> save -> deploy

Create Api Gateway

  • On Api Gateway Dashboard
  • Choose as API type the HTTP API -> Build
  • Add Integration -> select lambda
  • Select the region where yo created your lambda function
  • On Lambda function select your lambda function (in my case open-search-lambda-function)
  • API name : movies-api -> could be any.
  • Next
  • In Configure routes
  • Method : GET
  • Resource path : /search -> could be any, but consider this will be our endpoint in the url.
  • Integration target : should be our lambda function name.
  • Next
  • Leave Auto-deploy as set.
  • Next
  • Create
  • Once it has been created , go to Develop -> Cors Image description
  • Inside of cors go to configure
  • White an * in Access-Control-Allow-Origin and in Access-Control-Allow-Headers, and click on the Add button for both. Image description
  • Save

Set other configurations on the Lambda Function

  • Once the OpenSearch domain has successfully been created Image description
  • Inside our lambda function, go to configuration tab -> Environment variables Image description
  • Click on Edit -> Add Environment Variable
  • Key : DOMAIN_URL
  • Value : this url shoul be taken from OpenSearch Domain Endpoint Image description
  • But Very Important!!! remove https:// and the last /
  • For this example, my Value would be : search-mydomain-bmrj5t5f5tkh5jen3sn5gnzm6u.us-east-1.es.amazonaws.com.
  • Save

Configuring OpenSearch database from Dashboard

  • In our OpenSearch domain, click on OpenSearch Dashboards URL. Image description
  • This will open a new tab with the OpenSearch Console.
  • Yo will be asked to enter your username and password.
  • Select your tenant -> cancel
  • Click on the burger button -> OpenSearch Plugins -> Security
  • Go to Roles and click on all_access. Image description
  • Select the tab Mapped users
  • Click on the Manage mapping button Image description
  • Users -> select the user created on "Create OpenSearch Domain"
  • Backend roles -> in this area we must enter the lambda ARN role. This is located in Iam AWS service. Image description
  • Click on your lambda function role. And the ARN it there: Image description
  • Map users should be like this: Image description
  • Map
  • All_access should be like this: Image description

Populate database from local console

  • Now will try to insert a new element inside our database.
  • Open your local terminal (you should have installed the curl library)
  • In the next curl, please, provide your username and password. The domain endpoint is the OpenSearch Dashboards URL (Important: remove the /_dashboards at the end, but insert /movies/_doc/1).
curl -XPUT -u 'userName:userPassword' 'domain-endpoint/movies/_doc/1' -d '{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi"], "year": 1996, "actor": ["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}' -H 'Content-Type: application/json'
Enter fullscreen mode Exit fullscreen mode
  • If the return of this curl was unauthorized, please verify username and password provided.
  • If you receive data, it means you have added an item to the database
["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}' -H 'Content-Type: application/json'
Enter fullscreen mode Exit fullscreen mode
  • To verify if data has been added in the OpenSearch Dashboard, go to burger button -> Dashboard. You should see something like this: Image description
  • If you see this, congratulations. You have added data to your database. Now we should create and index, for OpenSearch to recognice table column names.
  • Click on Create index pattern.
  • Index name : movies* (this is the table name we provided at the url, when we added data to the DB).
  • Next step.
  • Create index pattern.
  • To see the data go to burger button -> Discover.1
  • You shoul see one hit.
  • Lets add more data. Create a file in your current working directory (I mean the adress where your console is located now) called "bulk.json" and paste inside the next information:
{"index":{"_id":1}}
{"Crawl Timestamp":"2019-05-29 12:32:00 +0000","Pageurl":"https://www.imdb.com/title/tt6791350/","Title":"Guardians of the Galaxy Vol. 3 (2021) - IMDb","Genres":"","Release Date":"","Movie Rating":"","Review Rating":0.0,"Movie Run Time":"","Plot":"Directed by James Gunn. With Elizabeth Debicki, Chris Pratt, Pom Klementieff, Dave Bautista. Plot unknown. Third installment of the 'Guardians of the Galaxy' franchise.","Cast":"","Language":"English","Filming Locations":"","Budget":"","Collection":"","Collection Date":""}
{"index":{"_id":2}}
{"Crawl Timestamp":"2019-05-29 12:30:33 +0000","Pageurl":"https://www.imdb.com/title/tt1228705/","Title":"Iron Man 2 (2010) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"7 May 2010 (USA)","Movie Rating":"PG-13","Review Rating":7.0,"Movie Run Time":"124 min","Plot":"Directed by Jon Favreau. With Robert Downey Jr., Mickey Rourke, Gwyneth Paltrow, Don Cheadle. With the world now aware of his identity as Iron Man, Tony Stark must contend with both his declining health and a vengeful mad man with ties to his father's legacy.","Cast":"Robert Downey Jr.|Mickey Rourke|Gwyneth Paltrow|Don Cheadle","Language":"English|French|Russian","Filming Locations":"D.C. Stages, 1360 East 6th Street, Downtown, Los Angeles, California, USA","Budget":"$200,000,000","Collection":"$623,933,331,","Collection Date":"19 August 2010"}
{"index":{"_id":3}}
{"Crawl Timestamp":"2019-05-29 12:31:05 +0000","Pageurl":"https://www.imdb.com/title/tt2015381/","Title":"Guardians of the Galaxy (2014) - IMDb","Genres":"Action|Adventure|Comedy|Sci-Fi","Release Date":"1 August 2014 (USA)","Movie Rating":"PG-13","Review Rating":8.1,"Movie Run Time":"121 min","Plot":"Directed by James Gunn. With Chris Pratt, Vin Diesel, Bradley Cooper, Zoe Saldana. A group of intergalactic criminals must pull together to stop a fanatical warrior with plans to purge the universe.","Cast":"Chris Pratt|Vin Diesel|Bradley Cooper|Zoe Saldana","Language":"English","Filming Locations":"Shepperton Studios, Shepperton, Surrey, England, UK","Budget":"$170,000,000","Collection":"$774,176,600","Collection Date":""}
{"index":{"_id":4}}
{"Crawl Timestamp":"2019-05-29 12:31:37 +0000","Pageurl":"https://www.imdb.com/title/tt1825683/","Title":"Black Panther (2018) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"16 February 2018 (USA)","Movie Rating":"PG-13","Review Rating":7.3,"Movie Run Time":"134 min","Plot":"Directed by Ryan Coogler. With Chadwick Boseman, Michael B. Jordan, Lupita Nyong'o, Danai Gurira. T'Challa, heir to the hidden but advanced kingdom of Wakanda, must step forward to lead his people into a new future and must confront a challenger from his country's past.","Cast":"Chadwick Boseman|Michael B. Jordan|Lupita Nyong'o|Danai Gurira","Language":"English|Swahili|Nama|Xhosa|Korean","Filming Locations":"Pinewood Atlanta Studios, 461 Sandy Creek Road, Fayetteville, Georgia, USA","Budget":"$200,000,000","Collection":"$1,347,071,259","Collection Date":""}
{"index":{"_id":5}}
{"Crawl Timestamp":"2019-05-29 12:30:49 +0000","Pageurl":"https://www.imdb.com/title/tt1300854/","Title":"Iron Man 3 (2013) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"3 May 2013 (USA)","Movie Rating":"PG-13","Review Rating":7.2,"Movie Run Time":"130 min","Plot":"Directed by Shane Black. With Robert Downey Jr., Guy Pearce, Gwyneth Paltrow, Don Cheadle. When Tony Stark's world is torn apart by a formidable terrorist called the Mandarin, he starts an odyssey of rebuilding and retribution.","Cast":"Robert Downey Jr.|Guy Pearce|Gwyneth Paltrow|Don Cheadle","Language":"English","Filming Locations":"Cary, North Carolina, USA","Budget":"$200,000,000","Collection":"$1,215,439,994","Collection Date":""}
{"index":{"_id":6}}
{"Crawl Timestamp":"2019-05-29 12:30:25 +0000","Pageurl":"https://www.imdb.com/title/tt0800080/","Title":"The Incredible Hulk (2008) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"13 June 2008 (USA)","Movie Rating":"PG-13","Review Rating":6.7,"Movie Run Time":"112 min","Plot":"Directed by Louis Leterrier. With Edward Norton, Liv Tyler, Tim Roth, William Hurt. Bruce Banner, a scientist on the run from the U.S. Government, must find a cure for the monster he turns into, whenever he loses his temper.","Cast":"Edward Norton|Liv Tyler|Tim Roth|William Hurt","Language":"English|Portuguese|Spanish","Filming Locations":"Cherry Street Bridge, Toronto, Ontario, Canada","Budget":"$150,000,000","Collection":"$263,427,551","Collection Date":""}
{"index":{"_id":7}}
{"Crawl Timestamp":"2019-05-29 12:31:49 +0000","Pageurl":"https://www.imdb.com/title/tt4154664/","Title":"Captain Marvel (2019) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"8 March 2019 (USA)","Movie Rating":"PG-13","Review Rating":7.1,"Movie Run Time":"123 min","Plot":"Directed by Anna Boden, Ryan Fleck. With Brie Larson, Samuel L. Jackson, Ben Mendelsohn, Jude Law. Carol Danvers becomes one of the universe's most powerful heroes when Earth is caught in the middle of a galactic war between two alien races.","Cast":"Brie Larson|Samuel L. Jackson|Ben Mendelsohn|Jude Law","Language":"English","Filming Locations":"Los Angeles, California, USA","Budget":"","Collection":"$1,126,318,317,","Collection Date":"20 May 2019"}
{"index":{"_id":8}}
{"Crawl Timestamp":"2019-05-29 12:30:21 +0000","Pageurl":"https://www.imdb.com/title/tt0371746/","Title":"Iron Man (2008) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"2 May 2008 (USA)","Movie Rating":"PG-13","Review Rating":7.9,"Movie Run Time":"126 min","Plot":"Directed by Jon Favreau. With Robert Downey Jr., Gwyneth Paltrow, Terrence Howard, Jeff Bridges. After being held captive in an Afghan cave, billionaire engineer Tony Stark creates a unique weaponized suit of armor to fight evil.","Cast":"Robert Downey Jr.|Gwyneth Paltrow|Terrence Howard|Jeff Bridges","Language":"English|Persian|Urdu|Arabic|Hungarian","Filming Locations":"Palmdale Regional Airport, Palmdale, California, USA","Budget":"$140,000,000","Collection":"$585,174,222,","Collection Date":"2 October 2008"}
{"index":{"_id":9}}
{"Crawl Timestamp":"2019-05-29 12:30:38 +0000","Pageurl":"https://www.imdb.com/title/tt0800369/","Title":"Thor (2011) - IMDb","Genres":"Action|Adventure|Fantasy|Sci-Fi","Release Date":"6 May 2011 (USA)","Movie Rating":"PG-13","Review Rating":7.0,"Movie Run Time":"115 min","Plot":"Directed by Kenneth Branagh. With Chris Hemsworth, Anthony Hopkins, Natalie Portman, Tom Hiddleston. The powerful but arrogant god Thor (Chris Hemsworth) is cast out of Asgard to live amongst humans in Midgard (Earth), where he soon becomes one of their finest defenders.","Cast":"Chris Hemsworth|Anthony Hopkins|Natalie Portman|Tom Hiddleston","Language":"English","Filming Locations":"Galisteo, New Mexico, USA","Budget":"$150,000,000","Collection":"$449,326,618","Collection Date":""}
{"index":{"_id":10}}
{"Crawl Timestamp":"2019-05-29 12:31:19 +0000","Pageurl":"https://www.imdb.com/title/tt3498820/","Title":"Captain America: Civil War (2016) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"6 May 2016 (USA)","Movie Rating":"PG-13","Review Rating":7.8,"Movie Run Time":"147 min","Plot":"Directed by Anthony Russo, Joe Russo. With Chris Evans, Robert Downey Jr., Scarlett Johansson, Sebastian Stan. Political involvement in the Avengers' affairs causes a rift between Captain America and Iron Man.","Cast":"Chris Evans|Robert Downey Jr.|Scarlett Johansson|Sebastian Stan","Language":"English|German|Xhosa|Russian|Romanian|Hindi","Filming Locations":"Atlanta, Georgia, USA","Budget":"$250,000,000","Collection":"$1,153,304,495","Collection Date":""}
{"index":{"_id":11}}
{"Crawl Timestamp":"2019-05-29 12:31:57 +0000","Pageurl":"https://www.imdb.com/title/tt6320628/","Title":"Spider-Man: Far from Home (2019) - IMDb","Genres":"","Release Date":"2 July 2019 (USA)","Movie Rating":"","Review Rating":7.5,"Movie Run Time":"","Plot":"Directed by Jon Watts. With Zendaya, Tom Holland, Marisa Tomei, Jake Gyllenhaal. Following the events of Avengers: Endgame, Spider-Man must step up to take on new threats in a world that has changed forever.","Cast":"","Language":"English","Filming Locations":"Hertfordshire, England, UK","Budget":"","Collection":"","Collection Date":""}
{"index":{"_id":12}}
{"Crawl Timestamp":"2019-05-29 12:31:41 +0000","Pageurl":"https://www.imdb.com/title/tt4154756/","Title":"Avengers: Infinity War (2018) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"27 April 2018 (USA)","Movie Rating":"PG-13","Review Rating":8.5,"Movie Run Time":"149 min","Plot":"Directed by Anthony Russo, Joe Russo. With Robert Downey Jr., Chris Hemsworth, Mark Ruffalo, Chris Evans. The Avengers and their allies must be willing to sacrifice all in an attempt to defeat the powerful Thanos before his blitz of devastation and ruin puts an end to the universe.","Cast":"Robert Downey Jr.|Chris Hemsworth|Mark Ruffalo|Chris Evans","Language":"English","Filming Locations":"Pinewood Atlanta Studios, 461 Sandy Creek Road, Fayetteville, Georgia, USA","Budget":"$321,000,000","Collection":"$2,048,709,917","Collection Date":""}
{"index":{"_id":13}}
{"Crawl Timestamp":"2019-05-29 12:31:28 +0000","Pageurl":"https://www.imdb.com/title/tt2250912/","Title":"Spider-Man: Homecoming (2017) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"7 July 2017 (USA)","Movie Rating":"PG-13","Review Rating":7.5,"Movie Run Time":"133 min","Plot":"Directed by Jon Watts. With Tom Holland, Michael Keaton, Robert Downey Jr., Marisa Tomei. Peter Parker balances his life as an ordinary high school student in Queens with his superhero alter-ego Spider-Man, and finds himself on the trail of a new menace prowling the skies of New York City.","Cast":"Tom Holland|Michael Keaton|Robert Downey Jr.|Marisa Tomei","Language":"English|Spanish","Filming Locations":"Atlanta, Georgia, USA","Budget":"$175,000,000","Collection":"$880,166,924","Collection Date":""}
{"index":{"_id":14}}
{"Crawl Timestamp":"2019-05-29 12:30:41 +0000","Pageurl":"https://www.imdb.com/title/tt0458339/","Title":"Captain America: The First Avenger (2011) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"22 July 2011 (USA)","Movie Rating":"PG-13","Review Rating":6.9,"Movie Run Time":"124 min","Plot":"Directed by Joe Johnston. With Chris Evans, Hugo Weaving, Samuel L. Jackson, Hayley Atwell. Steve Rogers, a rejected military soldier transforms into Captain America after taking a dose of a Super-Soldier serum. But being Captain America comes at a price as he attempts to take down a war monger and a terrorist organization.","Cast":"Chris Evans|Hugo Weaving|Samuel L. Jackson|Hayley Atwell","Language":"English|Norwegian|French","Filming Locations":"Stanley Dock, Liverpool, Merseyside, England, UK","Budget":"$140,000,000","Collection":"$370,569,774","Collection Date":""}
{"index":{"_id":15}}
{"Crawl Timestamp":"2019-05-29 12:31:53 +0000","Pageurl":"https://www.imdb.com/title/tt4154796/","Title":"Avengers: Endgame (2019) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"26 April 2019 (USA)","Movie Rating":"PG-13","Review Rating":8.8,"Movie Run Time":"181 min","Plot":"Directed by Anthony Russo, Joe Russo. With Robert Downey Jr., Chris Evans, Mark Ruffalo, Chris Hemsworth. After the devastating events of Avengers: Infinity War (2018), the universe is in ruins. With the help of remaining allies, the Avengers assemble once more in order to undo Thanos' actions and restore order to the universe.","Cast":"Robert Downey Jr.|Chris Evans|Mark Ruffalo|Chris Hemsworth","Language":"English|Japanese|Xhosa","Filming Locations":"Atlanta,Georgia,USA","Budget":"$356,000,000","Collection":"$2,681,988,528,","Collection Date":"26 May 2019"}
{"index":{"_id":16}}
{"Crawl Timestamp":"2019-05-29 12:31:44 +0000","Pageurl":"https://www.imdb.com/title/tt5095030/","Title":"Ant-Man and the Wasp (2018) - IMDb","Genres":"Action|Adventure|Comedy|Sci-Fi","Release Date":"6 July 2018 (USA)","Movie Rating":"PG-13","Review Rating":7.1,"Movie Run Time":"118 min","Plot":"Directed by Peyton Reed. With Paul Rudd, Evangeline Lilly, Michael Peña, Walton Goggins. As Scott Lang balances being both a Super Hero and a father, Hope van Dyne and Dr. Hank Pym present an urgent new mission that finds the Ant-Man fighting alongside The Wasp to uncover secrets from their past.","Cast":"Paul Rudd|Evangeline Lilly|Michael Peña|Walton Goggins","Language":"English|Spanish","Filming Locations":"Atlanta, Georgia, USA","Budget":"$162,000,000","Collection":"$622,674,139","Collection Date":""}
{"index":{"_id":7}}
{"Crawl Timestamp":"2019-05-29 12:31:14 +0000","Pageurl":"https://www.imdb.com/title/tt0478970/","Title":"Ant-Man (2015) - IMDb","Genres":"Action|Adventure|Comedy|Sci-Fi","Release Date":"17 July 2015 (USA)","Movie Rating":"PG-13","Review Rating":7.3,"Movie Run Time":"117 min","Plot":"Directed by Peyton Reed. With Paul Rudd, Michael Douglas, Corey Stoll, Evangeline Lilly. Armed with a super-suit with the astonishing ability to shrink in scale but increase in strength, cat burglar Scott Lang must embrace his inner hero and help his mentor, Dr. Hank Pym, plan and pull off a heist that will save the world.","Cast":"Paul Rudd|Michael Douglas|Corey Stoll|Evangeline Lilly","Language":"English","Filming Locations":"Pinewood Atlanta Studios, 461 Sandy Creek Road, Fayetteville, Georgia, USA","Budget":"$130,000,000","Collection":"$519,445,163","Collection Date":""}
{"index":{"_id":18}}
{"Crawl Timestamp":"2019-05-29 12:30:51 +0000","Pageurl":"https://www.imdb.com/title/tt1981115/","Title":"Thor: The Dark World (2013) - IMDb","Genres":"Action|Adventure|Fantasy","Release Date":"8 November 2013 (USA)","Movie Rating":"PG-13","Review Rating":6.9,"Movie Run Time":"112 min","Plot":"Directed by Alan Taylor. With Chris Hemsworth, Natalie Portman, Tom Hiddleston, Stellan Skarsgård. When Dr. Jane Foster (Natalie Portman) gets cursed with a powerful entity known as the Aether, Thor is heralded of the cosmic event known as the Convergence and the genocidal Dark Elves.","Cast":"Chris Hemsworth|Natalie Portman|Tom Hiddleston|Stellan Skarsgård","Language":"English","Filming Locations":"Shepperton Studios, Shepperton, Surrey, England, UK","Budget":"$170,000,000","Collection":"$644,783,140","Collection Date":""}
{"index":{"_id":19}}
{"Crawl Timestamp":"2019-05-29 12:30:55 +0000","Pageurl":"https://www.imdb.com/title/tt1843866/","Title":"Captain America: The Winter Soldier (2014) - IMDb","Genres":"Action|Adventure|Sci-Fi|Thriller","Release Date":"4 April 2014 (USA)","Movie Rating":"PG-13","Review Rating":7.8,"Movie Run Time":"136 min","Plot":"Directed by Anthony Russo, Joe Russo. With Chris Evans, Samuel L. Jackson, Scarlett Johansson, Robert Redford. As Steve Rogers struggles to embrace his role in the modern world, he teams up with a fellow Avenger and S.H.I.E.L.D agent, Black Widow, to battle a new threat from history: an assassin known as the Winter Soldier.","Cast":"Chris Evans|Samuel L. Jackson|Scarlett Johansson|Robert Redford","Language":"English|French","Filming Locations":"Los Angeles, California, USA","Budget":"$170,000,000","Collection":"$714,766,572","Collection Date":""}
{"index":{"_id":20}}
{"Crawl Timestamp":"2019-05-29 12:31:10 +0000","Pageurl":"https://www.imdb.com/title/tt2395427/","Title":"Avengers: Age of Ultron (2015) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"1 May 2015 (USA)","Movie Rating":"PG-13","Review Rating":7.3,"Movie Run Time":"141 min","Plot":"Directed by Joss Whedon. With Robert Downey Jr., Chris Evans, Mark Ruffalo, Chris Hemsworth. When Tony Stark and Bruce Banner try to jump-start a dormant peacekeeping program called Ultron, things go horribly wrong and it's up to Earth's mightiest heroes to stop the villainous Ultron from enacting his terrible plan.","Cast":"Robert Downey Jr.|Chris Evans|Mark Ruffalo|Chris Hemsworth","Language":"English|Korean","Filming Locations":"Shepperton Studios, Shepperton, Surrey, England, UK","Budget":"$250,000,000","Collection":"$1,405,413,868","Collection Date":""}
{"index":{"_id":21}}
{"Crawl Timestamp":"2019-05-29 12:31:25 +0000","Pageurl":"https://www.imdb.com/title/tt3896198/","Title":"Guardians of the Galaxy Vol. 2 (2017) - IMDb","Genres":"Action|Adventure|Comedy|Sci-Fi","Release Date":"5 May 2017 (USA)","Movie Rating":"PG-13","Review Rating":7.7,"Movie Run Time":"136 min","Plot":"Directed by James Gunn. With Chris Pratt, Zoe Saldana, Dave Bautista, Vin Diesel. The Guardians struggle to keep together as a team while dealing with their personal family issues, notably Star-Lord's encounter with his father the ambitious celestial being Ego.","Cast":"Chris Pratt|Zoe Saldana|Dave Bautista|Vin Diesel","Language":"English","Filming Locations":"Atlanta, Georgia, USA","Budget":"$200,000,000","Collection":"$863,756,051","Collection Date":""}
{"index":{"_id":22}}
{"Crawl Timestamp":"2019-05-29 12:31:33 +0000","Pageurl":"https://www.imdb.com/title/tt3501632/","Title":"Thor: Ragnarok (2017) - IMDb","Genres":"Action|Adventure|Comedy|Fantasy|Sci-Fi","Release Date":"3 November 2017 (USA)","Movie Rating":"PG-13","Review Rating":7.9,"Movie Run Time":"130 min","Plot":"Directed by Taika Waititi. With Chris Hemsworth, Tom Hiddleston, Cate Blanchett, Mark Ruffalo. Thor (Chris Hemsworth) is imprisoned on the planet Sakaar, and must race against time to return to Asgard and stop Ragnarök, the destruction of his world, at the hands of the powerful and ruthless villain Hela (Cate Blanchett).","Cast":"Chris Hemsworth|Tom Hiddleston|Cate Blanchett|Mark Ruffalo","Language":"English","Filming Locations":"Village Roadshow Studios, Oxenford, Queensland, Australia","Budget":"$180,000,000","Collection":"$853,977,126","Collection Date":""}
{"index":{"_id":23}}
{"Crawl Timestamp":"2019-05-29 12:31:22 +0000","Pageurl":"https://www.imdb.com/title/tt1211837/","Title":"Doctor Strange (2016) - IMDb","Genres":"Action|Adventure|Fantasy|Sci-Fi","Release Date":"4 November 2016 (USA)","Movie Rating":"PG-13","Review Rating":7.5,"Movie Run Time":"115 min","Plot":"Directed by Scott Derrickson. With Benedict Cumberbatch, Chiwetel Ejiofor, Rachel McAdams, Benedict Wong. While on a journey of physical and spiritual healing, a brilliant neurosurgeon is drawn into the world of the mystic arts.","Cast":"Benedict Cumberbatch|Chiwetel Ejiofor|Rachel McAdams|Benedict Wong","Language":"English","Filming Locations":"New York City, New York, USA","Budget":"$165,000,000","Collection":"$677,718,395","Collection Date":""}
{"index":{"_id":0}}
{"Crawl Timestamp":"2019-05-29 12:30:44 +0000","Pageurl":"https://www.imdb.com/title/tt0848228/","Title":"The Avengers (2012) - IMDb","Genres":"Action|Adventure|Sci-Fi","Release Date":"4 May 2012 (USA)","Movie Rating":"PG-13","Review Rating":8.1,"Movie Run Time":"143 min","Plot":"Directed by Joss Whedon. With Robert Downey Jr., Chris Evans, Scarlett Johansson, Jeremy Renner. Earth's mightiest heroes must come together and learn to fight as a team if they are going to stop the mischievous Loki and his alien army from enslaving humanity.","Cast":"Robert Downey Jr.|Chris Evans|Scarlett Johansson|Jeremy Renner","Language":"English|Russian|Hindi","Filming Locations":"Pittsburgh, Pennsylvania, USA","Budget":"$220,000,000","Collection":"$1,519,557,910","Collection Date":""}
Enter fullscreen mode Exit fullscreen mode
  • Now in your terminal run the next curl:
curl -XPOST -u 'userName:userPassword' 'domain-endpoint/movies/_bulk' --data-binary @bulk.json -H 'Content-Type: application/json'
Enter fullscreen mode Exit fullscreen mode
  • If it runned ok, you should see all the data you added in the terminal, but too on the OpenSearch Dashboard -> Discover.

Verify endpoint is working

  • Go to you lambda funcion - Configuration -> Triggers. There you can see your endpoint. Image description
  • Click on it and the endpoint should be /search?key=a
  • If you see a big JSON, congratulations, you have configured correctly you API+Lambda+OpenSeach stack!!!.

Resources:

AntStack.io
AWS

Oldest comments (2)

Collapse
 
jeel007 profile image
Tushar

Hello.. Can we use the same deployment if our OpenSearch is deployed in private subnets?

Collapse
 
jeel007 profile image
Tushar

Solved it but huge thanks, this solution blog has shown the path.