Basico 0.2 released

2017-06-23 t00mNo Comments

Hi,

I am proud to announce Basico 0.2

It has been a long time since I released version 0.1 and there have been many changes. And many challenges.

Core internals

First of challenges was to allow Basico to download SAP Notes again. There was a problem of compatibility between Selenium webdriver and Firefox, therefore Basico stopped working suddenly. Months later, Selenium webdriver was updated and, by using the Gecko webdriver, it worked again but… Basico was unable to login in Launchpad using a S-User and password :(

Finally, I came up with the idea of using Single Sign-On with SAP Passports. And it works out very well. Just apply for your SAP Passport, add it to your firefox browser and trust it (see INSTALL page).

After this change I’ve been still unable to setup Selenium’s webdriver properly for working with threads, so SAP Notes are downloaded sequentially, one by one.

User Interface

In the UI side there has been a lot of improvements.

Views

Now, you can browse your SAP Notes using many views: by tasks, component, component code, component description, bookmarks, categories, crhonologic order, priority or type.

The most interesting one is Tasks. You can select one or more SAP Notes and group them in a task. So, for example, if you are working in an SAP BW System, you can add all related SAP Notes that you use in a task:

Another view very useful is Chronological order. Here you can check your day to day and know what you were working with:

Operations

When you select one or more SAP Notes, Operations menu is activated. Then, you can choose to browse selected SAP Notes, bookmark them , manage task(s) they belong to, export them or delete them.

Also, from this menu, you can export your SAP Notes by choosing one of the following formats:

JSON: share your SAP Notes with all metadata
CSV: the right format to include in your reports or documents
Plain text: it only saves SAP Note Id(s)

Download / Import SAP Notes

Here, you can import a JSON file with SAP Notes or you can download them from Launchpad.

Search & Filter

By clicking in Search button you enable the search entry to search in all database and, optionally, filter by one of the available properties:

SAP Note metadata details

From fingerprint button you can see all details for selected SAP Note.

A program execution worths more than one thousand words. Give it a try ;-)

Get Firefox profile directory with Python (Linux and Windows)

2017-03-272017-03-28 t00mNo Comments

Piece of code used in Basico (util.py module) to get the default user’s Firefox profile directory.

This profile must have the SAP Passport imported in the Certificate store.

import sys

def get_firefox_profile_dir():
    if sys.platform in ['linux', 'linux2']:
        import subprocess
        cmd = "ls -d /home/$USER/.mozilla/firefox/*.default/"
        p = subprocess.Popen([cmd], shell=True, stdout=subprocess.PIPE)
        FF_PRF_DIR = p.communicate()[0][0:-2]
        FF_PRF_DIR_DEFAULT = str(FF_PRF_DIR,'utf-8')
    elif sys.platform == 'win32':
        import os
        import glob
        APPDATA = os.getenv('APPDATA')
        FF_PRF_DIR = "%s\\Mozilla\\Firefox\\Profiles\\" % APPDATA
        PATTERN = FF_PRF_DIR + "*default*"
        FF_PRF_DIR_DEFAULT = glob.glob(PATTERN)[0]

    return FF_PRF_DIR_DEFAULT

print (get_firefox_profile_dir())

import sys

def get_firefox_profile_dir():

if sys.platform in ['linux', 'linux2']:

import subprocess

cmd = "ls -d /home/$USER/.mozilla/firefox/*.default/"

p = subprocess.Popen([cmd], shell=True, stdout=subprocess.PIPE)

FF_PRF_DIR = p.communicate()[0][0:-2]

FF_PRF_DIR_DEFAULT = str(FF_PRF_DIR,'utf-8')

elif sys.platform == 'win32':

import os

import glob

APPDATA = os.getenv('APPDATA')

FF_PRF_DIR = "%s\\Mozilla\\Firefox\\Profiles\\" % APPDATA

PATTERN = FF_PRF_DIR + "*default*"

FF_PRF_DIR_DEFAULT = glob.glob(PATTERN)[0]

return FF_PRF_DIR_DEFAULT

print (get_firefox_profile_dir())

Basico: a SAP Notes Manager for SAP Consultants

2016-05-312016-05-31 t00m1 Comment

Basico

The main goal for this application is to assist you (the SAP Consultant), to download the SAP Notes you are interested in, categorize them by tasks and, find them quickly. Another goal is to allow you to share SAP Notes with other fellows.

All of it started as an experiment about how to download SAP Notes programatically but quickly became in a small application to manage SAP Notes.

Please, if you are looking for something more serious and reliable go to Launchpad@SAP.

Disclaimer

This software is not linked, affiliated, related or endorsed in any way by any division or subsidiary of SAP® AG.

I always thought that when you browse a SAP Note, the source code would have plain HTML with something of Javascript. I was totally wrong. Almost all source code is Javascript, which is imposible to parse with web scrapper tools.

So, I asked in SCN forums how to download SAP Notes programatically with Python.

Months later I got the first answer which, in turn, became in a deep investigation about available python libraries to work with OData services. Until that moment, OData was another buzzword for me. I didn’t know anything else about this protocol.

In short, you have to build and send a query. Then, the server returns the OData XML. Easy, isn’t it? Well, it is not. It’s been a nightmare for me the last months.

The new ‘Launchpad‘ service from SAP let us consume OData services over HTTPS.

For example, if you want to get the SAP Note 1, you can use this URL and you’ll get the OData XML file.

I only like to develop in Linux environments so the python library should run in Linux and, if possible, be cross-platform. And, of course, be Open Source. This is what I found:

Pyslet: OData v2 with both client and server capabilities. Cross-platform. Limited HTTPS support based on certificates. Open-Source (BSD 3-Clause License).
ODataPy: OData v4 client. Only for Windows platforms. Open-Source (MIT License).
Auth0 SAP OData API: experimental addon to work with SAP OData. Commercial, not open-source.

At first sight, the best option was Pyslet: it’s a Python library, runs in Linux, it is open source and, has support for HTTPS. I’ve been unable to use Pyslet with certificates, though. I’ve tried again and again but without luck. Sadly, I had to discard this option.

So now what?

The same person who answered me in the forum suggested me to look for something similiar to HTMLUnit in Java but for Python. From its webpage:

“””
HtmlUnit is a “GUI-Less browser for Java programs”. It models HTML documents and provides an API that allows you to invoke pages, fill out forms, click links, etc… just like you do in your “normal” browser.
“””

Well, two minutes later, I found it. It’s Selenium: one of the best tools I have ever seen.

Selenium automates browsers.

Selenium is a portable software testing framework for web applications. It has bindings for Python among others programming languages. The tests can then be run against most modern web browsers (Firefox, Chrome, IE and, PhantomJS -a headless browser-). Selenium deploys on Windows, Linux, and Macintosh platforms. It is open-source software, released under the Apache 2.0 license.

This is a little example of how to download a SAP Note with Selenium and Python:
(Change S_USER and S_PASS with your S-User login data)

#!/usr/bin/python
# -*- coding: utf-8 -*-
# File: sapnote.py

from selenium import webdriver

LOGIN_PAGE_URL = "https://accounts.sap.com"
LOGOUT_PAGE_URL = "https://accounts.sap.com/ui/logout"
ODATA_SERVICE = "https://launchpad.support.sap.com/services/odata/svt/snogwscorr/"
ODATA_QUERY = "TrunkSet(SapNotesNumber='%s',Version='0',Language='E')"
ODATA_NOTE_URL = ODATA_SERVICE + ODATA_QUERY
S_USER = "SXXXXXXXXXX"
S_PASS = "XXXXXXXXXXX"

# SSO SAP Login
driver = webdriver.Firefox()
driver.get(LOGIN_PAGE_URL)
username = driver.find_element_by_id('j_username')
username.send_keys(S_USER)
password = driver.find_element_by_id('j_password')
password.send_keys(S_PASS)
login = driver.find_element_by_id('logOnFormSubmit')
login.click()

# Get OData XML for SAP Note 1
driver.get(ODATA_NOTE_URL % '1')
sapnote = open('sapnote.xml', 'w')
sapnote.write(driver.page_source)
sapnote.close()

# Logout
driver.get(LOGOUT_PAGE_URL)

#!/usr/bin/python

# -*- coding: utf-8 -*-

# File: sapnote.py

from selenium import webdriver

LOGIN_PAGE_URL = "https://accounts.sap.com"

LOGOUT_PAGE_URL = "https://accounts.sap.com/ui/logout"

ODATA_SERVICE = "https://launchpad.support.sap.com/services/odata/svt/snogwscorr/"

ODATA_QUERY = "TrunkSet(SapNotesNumber='%s',Version='0',Language='E')"

ODATA_NOTE_URL = ODATA_SERVICE + ODATA_QUERY

S_USER = "SXXXXXXXXXX"

S_PASS = "XXXXXXXXXXX"

# SSO SAP Login

driver = webdriver.Firefox()

driver.get(LOGIN_PAGE_URL)

username = driver.find_element_by_id('j_username')

username.send_keys(S_USER)

password = driver.find_element_by_id('j_password')

password.send_keys(S_PASS)

# Get OData XML for SAP Note 1

driver.get(ODATA_NOTE_URL % '1')

sapnote = open('sapnote.xml', 'w')

sapnote.write(driver.page_source)

sapnote.close()

# Logout

driver.get(LOGOUT_PAGE_URL)

At this development stage (basico 0.1), the only browser supported is Firefox.

Another excelent Python library is Universal Feed Parser (aka feedparser). It helped me to extract all the necessary info from OData XML.

This Python module download and parse syndicated feeds (it can handle RSS 0.90, Netscape RSS 0.91, Userland RSS 0.91, RSS 0.92, RSS 0.93, RSS 0.94, RSS 1.0, RSS 2.0, Atom 0.3, Atom 1.0, and CDF feeds. It also parses several popular extension modules, including Dublin Core and Apple’s iTunes extensions.

Summing up, I can download SAP Notes and, I can parse them. So the next question is: how should I store them? First, I thought about creating a SQLite database because it is quick and easy. But how could share my notes with other people? I needed something easiest to handle. Up to now the best answer is JSON.

JSON (JavaScript Object Notation) is a syntax for storing and exchanging data, easier-to-use alternative to XML, is text only and, can be read and used as a data format by any programming language.

Morover, I can export a dictionary to JSON and, viceversa, I can import JSON data to a dictionary in Python:

import json

# Export to JSON format
sapnotes = {} # Dictionary
fnotes = open('sapnotes.json', 'w')
json.dump(saponotes, fnotes)
fnotes.close()


# Import from JSON
fnotes = 'saponotes.json'
with open(fnotes, 'r') as fp:
    sapnotes = json.load(fp)

import json

# Export to JSON format

sapnotes = {} # Dictionary

fnotes = open('sapnotes.json', 'w')

json.dump(saponotes, fnotes)

fnotes.close()

# Import from JSON

fnotes = 'saponotes.json'

with open(fnotes, 'r') as fp:

sapnotes = json.load(fp)

Last step: build a GUI around all this concepts with the following prerequsites:

It must be easy to use by the end user.
It must be cross-platform
It must be easy to develop

The answer for this question is GTK+. GTK+, or the GIMP Toolkit, is a multi-platform toolkit for creating graphical user interfaces. Offering a complete set of widgets, GTK+ is suitable for projects ranging from small one-off tools to complete application suites.

So here we are:

Please, read this page in order to install Basico.

Hope you can enjoy it!