PowerShell – Using the New York Times Semantic Web APIs

December 4, 2011

in JSON,New York Times,PowerShell,PowerShell V3,Semantic Data

Mark Watson posted, Using the New York Times Semantic Web APIs. In it, he uses Clojure , AllegroGraph 4, Star Dog RDF repositories and his own NLP code.

After requesting API Key from the New York Times, I worked up this reusable query in PowerShell. Time for more exploration.

Note: This PowerShell code only works on Version 3. I am using the new cmdlet Invoke-RestMethod and the simplified Where syntax.

Get-SemanticNYT "obama" |            
    Get-SemanticNYTArticles |             
    where links |             
        select -ExpandProperty article_list |             
        select -ExpandProperty results |             
        select Title, Date, Body | Out-GridView

Injecting a GUI into the Command Line

Out-GridView is one of the cmdlets delivered with PowerShell.

PowerShell Code

PowerShell’s parameter binding saves time and effort. Specifying ValueFromPipelineByPropertyName, lets me quickly pipe results from one Web API to the other. Letting PowerShell do the work of binding the output from one function to the other. This lets me focus on problem solving.

Invoke-RestMethod does all the heavy lifting when interacting with the Web, (Invoke-RestMethod $uri).results:

Hits the endpoint
Retrieves the JSON
Converts the JSON to PowerShell objects

function Get-SemanticNYT {            
                
    param($query = "obama")            
            
    $uri = "http://api.nytimes.com/svc/semantic/v2/"+            
        "concept/search.json?query=$query&api-key=$apiKey"            
            
    (Invoke-RestMethod $uri).results            
}            
            
function Get-SemanticNYTArticles {            
                
    param(            
        [Parameter(ValueFromPipelineByPropertyName)]            
        $concept_name,            
        [Parameter(ValueFromPipelineByPropertyName)]            
        $concept_type            
    )            
            
    process {            
      $uri = "http://api.nytimes.com/svc/semantic/v2/" +            
      "concept/name/$concept_type/$concept_name.json?&" +            
      "fields=all&api-key=$apiKey"            
            
      (Invoke-RestMethod $uri).results            
    }            
}

{ 1 trackback }

PowerShell – Using the New York Times Semantic Web APIs | Sergio's Curation Powershell & Security | Scoop.it: 12.05.11 at 4:49 am

{ 2 comments… read them below or add one }

Stan Littlefield 12.06.11 at 2:44 pm

Very nice and concise. Is there any way to quickly determine the keywords, i.e. Title, Date, Body from a given JSON query? I know in this case it is format= but in most cases you have to look at a result set first, then re-write the query for specific data.

Doug Finke 12.06.11 at 3:00 pm

Thanks for the comment Stan. If I understand what you are asking, I can replace the last select with this:

Get-Member -MemberType *Property

Which prints:


   TypeName: Selected.System.Management.Automation.PSCustomObject

Name  MemberType   Definition                                                                                                                                    
----  ----------   ----------                                                                                                                                    
body  NoteProperty System.String body=To the Editor
date  NoteProperty System.String date=20111128                                                                                                                   
title NoteProperty System.String title=LETTER; Prot
url   NoteProperty System.String url=http://www.nyt

The following extracts and builds up keywords:

$result = Get-SemanticNYT "obama" |
    Get-SemanticNYTArticles | 
    where links | 
        select -ExpandProperty article_list | 
        select -ExpandProperty results 

( @($result)[0] | 
     Get-Member -MemberType *Property  | 
     select -ExpandProperty name) -join ", "