Čeština
English
  • Vítejte na stránkách NLP Centra!
  • Zapojte se do vývoje softwarových nástrojů!
  • Analýza přirozeného jazyka
  • Vyzkoušejte si korpusy o velikosti knihoven online!
  • Studujte jednu ze specializací!
  • Členové laboratoře

PV277 Programming Applications for Social Robots

[ previous part ] [ next part ]

How to get info from web?

Remember that Python usage in Pepper robots is limited to Python 2 and old versions of packages. It is possible to install some (small) Python packages via pip --user to /home/nao but the space is limited.

If you want to test without the real robot, you can run the Python 2 code (in or outside Choregraphe) using the virtual robot - see below.

Use website API

If API exists, you can usually use JSON data after send the right parameters. Example to get weather data.

import requests
url = "http://api.openweathermap.org/data/2.5/weather"

params = {'q': 'Brno',
          'units': 'metric',
          'lang': 'cz'}

r = requests.get(url=url, params=params)
data = r.json()

print(data['main']['temp'])

(Full example forecast.py )

Parse webpage data

If no API is provided, you can download and parse the webpage.

from bs4 import BeautifulSoup
import requests
page = requests.get(url).text
soup = BeautifulSoup(page, features="lxml")
results = soup.findAll('h3', attrs={'class':'article-title'})

(Full example news.py)

In virtual robot, if you get a runtime error: No module named bs4, you may need to install the package into the Choregraphe environtment via:

sudo pip2 install --target='/opt/Softbank Robotics/Choregraphe Suite 2.5/lib/python2.7/site-packages' \
    bs4

Use pip instead of pip2 in older linux systems. Beware that the version of just installed bs4 package may be different from the version on the real robot.

A safe option to include new packages is to install the package (e.g. yourmodule) in the application directory (e.g. lib) and import it like:

def __init__(self):
        GeneratedClass.__init__(self)
        self.folderName = os.path.join(self.behaviorAbsolutePath(), "../lib")

def onInput_onStart(self):
        if self.folderName not in sys.path:
            sys.path.append(self.folderName)
        import yourmodule
        # ... use yourmodule
        if self.folderName in sys.path:
            sys.path.remove(self.folderName)
        self.onStopped() #activate the output of the box

How to integrate info with robot?

There are 2 basic options:

  1. use simple dialog variables as input and say the results directly in the Python script
  2. use complex service (=a separate Python class/script) which fully integrates in the Pepper API

These options are detailed below.

Option 1 - Dialog variables

In the dialog topic file, set a value of a variable, use the variable in a Python script in the robot's behavior. See the example application below for details.

concept:(team) [sparta spartě slavie slavii]
u:(["Můj [oblíbený nejoblíbenější] tým je" "Fandím {týmu}"] _~team) $team=$1

First word after sentence is available in $1 and passed to the script as team variable.

def onInput_onStart(self, team):
    self.team_name = 'Slavia'
    if ((team.lower().startswith('spart')):
        self.team_name = 'Sparta'

Use TTS in script

# find out self.team_points for self.team_name from web
sentence = self.team_name + " má " + str(self.team_points) + " bodů"
self.tts.say(sentence)

Option 2 - Run script as service and parse service output

Specify service script in manifest.xml (here rozvrh = schedule):

<services>
    <service autorun="true" execStart="/usr/bin/python2 scripts/rozvrh.py" name="rozvrh" />
</services>

In dialog, detect variables, pass them to service, and parse the call result to say answer:

u:(rozvrh) Pro jakou místnost bys chtěl znát rozvrh?
    u1:({pro} {místnost učebnu} _~letter _~number) Podívám se na rozvrh pro $1 $2 ^call(Rozvrh.get_current_lesson($1, $2)) \pau=500\
    c1:(In _* teaches _* course _*) V místnosti $1 je právě $3 s vyučujícím, který se jmenuje $2.
    c1:(In _* currently _*) V místnosti $1 je právě $2

Note that in c1: we can use _* since this regular expression is matched against text from the service, not against human speech. The text can be in English (or even in non-language), the answer to humans is then specified in the concrete dialog topic, e.g. in Czech. Unfortunately, the documentation is not clear in what can and what cannot be matched by _* here, tests are needed with real data.

u1 detects letter and number (concepts defined earlier in dialog) and calls get_current_lesson(). This function return strings, that are parsed in dialog and answer is translated to each locale.

def get_current_lesson(self, letter, number):
    room = letter + str(number)
    lesson = self.rozvrh.find_current_lesson(room)
    if lesson:
        if lesson.teacher:
            teacher = lesson.teacher.split(' ')[-1]
            return "In %s teaches %s course %s" % (lesson.room, teacher, lesson.name)
         else:
            return "In %s currently %s" % (lesson.room, lesson.name)

Run script as service and use TTS in script

Detect variables in dialog, pass them to script and let script use TTS to say answer.

u:("[řekni ukaž zobraz najdi] {mi} [odjezdy spoje] ze zastávky _~station_name")
    ^call(DialogKordisbot.say_answer1($1))

Dialog detects word with station name and calls say_answer1(). No answer is passed back, answer is said directly in script.

def say_answer1(self, station):
    #get data finalDepartures
    answer_msg = "First line {} goes to {} at {}, second line {} goes to {} at {}, third line {} goes to {} at {}".format(finalDepartures[0][0], finalDepartures[0][1], finalDepartures[0][2], finalDepartures[1][0], finalDepartures[1][1], finalDepartures[1][2], finalDepartures[2][0], finalDepartures[2][1], finalDepartures[2][2])

    self.s.ALAnimatedSpeech.say(answer_msg)

Testing a service with virtual robot

A service can be tested in the virtual robot settings.

  • prepare an app with the service to test - you may start with the examples below or use the templater tool robot-jumpstarter (use a new name for your service):
    ssh aurora
    cd /nlp/projekty/pepper/web/robot-jumpstarter
    python jumpstart.py python-service tweety-service TweetyService
    cp -a output/tweety-service ~/pepper/
    
    Here tweety-service is the directory name of the application (and the app name), TweetyService is the API name of your service as it will be called from Python.
  • copy the directory with the service app (e.g. ~/pepper/tweety-service) to your computer, where you run Choregraphe. All the following steps are done on this computer, not remotely via SSH.
  • start Choregraphe and run the virtual robot
  • find out the port number of your virtual robot: click
    and remember the IP and port from the table:

    Here the virtual robot's address is localhost:34121
  • connecting to the Choregraphe virtual robot seems not to work in Windows environment, only in Linux. For Windows, the virtual robot can be run via the NAOqi binary:
    c:/Program\ Files/Softbank\ Robotics/Choregraphe\ Suite\ 2.5/bin/naoqi-bin
    
    with the output saying
    [I] 1650614881.924666 23993 qimessaging.servicedirectory: ServiceDirectory listener created on tcp://127.0.0.1:9559
    [I] 1650614881.925642 23993 qimessaging.transportserver: TransportServer will listen on: tcp://127.0.0.1:9559
    [I] 1650614881.927379 23993 qimessaging.servicedirectory: Registered Service "ServiceDirectory" (#1)
    ...
    
    Here the robot's URL is 127.0.0.1:9559.
  • run your service in the virtual robot (Choregraphe must still be running, of course):
    cd ~/pepper/tweety-service/app
    python2 scripts/tweetyservice.py --qi-url localhost:34121
    
    The output should look like:
    [I] 1584373059.579170 4749 qimessaging.session: Session listener created on tcp://0.0.0.0:0
    [I] 1584373059.579385 4749 qimessaging.transportserver: TransportServer will listen on: tcp://192.168.1.2:36873
    [I] 1584373059.579394 4749 qimessaging.transportserver: TransportServer will listen on: tcp://127.0.0.1:36873
    

Keep this service running (in the background or foreground, i.e. use other shell/window for the qicli commands).

  • now you can communicate with your running service in the same way as with all other API services on the real robot:
    • use ^call(TweetyService.get()) in a dialog
    • direct call in a Python Box code:
      def __init__(self):
         self.tweety = ALProxy('TweetyService')
      
      def onInput_onStart(self):
          self.tts.say("číslo {}".format(self.tweety.get()))
          self.onStopped() #activate the output of the box
      
    • use the qicli command line tool:
      alias qicli='"/opt/Softbank Robotics/Choregraphe Suite 2.5/bin/qicli" --qi-url localhost:34121'
      qicli info TweetyService
      qicli call TweetyService.get
      qicli call TweetyService.set 2
      qicli call TweetyService.get
      
      with the output
      099 [TweetyService]
        * Info:
         machine   f883b92e-3a87-44e5-aa70-4a3b78f4d937
         process   4749
         endpoints tcp://192.168.1.2:36873
                   tcp://127.0.0.1:36873
        * Methods:
         100 get   Int8 ()
         101 reset Void ()
         102 set   Void (Int8)
         103 stop  Void ()
      -------------------------------------------------------
      0
      null
      2
      
    • if the service uses Python packages from the Softbank site-packages, the service needs before the first import:
      import sys
      sys.path.append('/opt/Softbank Robotics/Choregraphe Suite 2.5/lib/python2.7/site-packages')
      

Example app

This application implements the option 1 of the two integration solutions mentioned above, i.e. it does not use a service, just simple dialog variables with standard Choregraphe box flow.

  • Create new project in Choregraphe.
  • As usual, add Czech in project Properties.
  • Add Set Language box and select Czech. Warnings may appear when running this box on the virtual robot (ALSpeechRecognition is not available ...) but these are harmless now.
  • Right click the free area -> Create a new box -> Dialog...
  • in the Dialog -> Add Topic - choose Czech and Add to the package content as collaborative dialog (allows to start the dialog just by talking to the robot)
  • Connect onStart -> Set Language -> Dialog
  • Right click dialog box -> Edit box. Find Outputs and click on + button. Name: country, Type: string, Nature: punctual

  • In Project files double click on virustat_czc.top and enter
    topic: ~virustat()
    language: czc
    
    u:(ahoj) ahoj, pro který stát tě zajímá statistika?
        u1:(Česko) dobrá, česko $country=Czechia
        u1:(Německo) dobrá, německo $country=Germany
    
  • Value of variable $country will be sent to the output named country.
  • Right click the free area -> Create a new box -> Python...
  • Find Inputs and click on the Edit icon next to onStart. set Type to String.
  • Find Outputs and click on + button. Name: returnStat, Type: string, Nature: punctual
  • Connect country output of the dialog with onStart input of the Python script.
  • Add Say Text box and connect returnStat output of the Python script with onStart input of Say Text box.
  • Connect onStopped output of Say Text box with onStopped.

  • Double-click the Python script to edit the script, and exchange autogenerated script with the following:
    class MyClass(GeneratedClass):
        def __init__(self):
            GeneratedClass.__init__(self)
    
        def onLoad(self):
            #put initialization code here
            pass
    
        def onUnload(self):
            #put clean-up code here
            pass
    
        def onInput_onStart(self, country):
            #self.onStopped() #activate the output of the box
            self.logger.info("start")
            self.logger.info(self.getStat(country))
            self.returnStat(self.getStat(country))
    
        def onInput_onStop(self):
            self.onUnload() #it is recommended to reuse the clean-up as the box is stopped
            self.onStopped() #activate the output of the box
    
        def getStat(self, country):
            import urllib2
            url = 'https://opendata.arcgis.com/datasets/bbb2e4f589ba40d692fab712ae37b9ac_2.csv'
            response = urllib2.urlopen(url)
            csv = response.read()
            for line in csv.splitlines():
                data = line.split(',')
                if data[3] == country:
                    return 'Počet nakažených je '+data[7]
    
  • Download the sample application (virustat.zip), unzip it and open in Choregraphe.

Other example applications

Explore example applications in /nlp/projekty/pepper/myapps:

  • kordisbot, Brno traffic dialog (runs own service DialogKordisbot, needs to install BeautifulSoup, reuqests and pyopenssl in the service environment)
  • sport_bot, dialog about Premiere League results
  • rozvrh, dialog about the faculty course schedule


[ previous part ] [ next part ]