Even though Linux has a lot of commands, sometimes it is necessary to write simple (and some not so simple) scripts when we have to automate jobs.

The Bash shell is capable of running scripts and it also possible to write scripts in the various programming languages that exist in Linux (such as AWK, Python and Ruby).

The following scripts need to be written and tested:

• booklist
  Manage a book database: print, insert, select, delete, update
• compareperm
  Check whether two files have the same permissions
  Usage: compareperm file1 file2
• findhardlinks
  Find links (not symbolic) to a given file
  Usage: findhardlinks file [directory]
• largestfile
  prints the largest file in a directory
  Usage: largestfile directory
• summarise
  prints number of normal files, directories and symbolic links
  summarise directory. Two versions of the script can be written: one that does three passes through the files and can be slow and a second version which only does one pass through the files and, consequently, is quicker.
• anagrams
  finds and prints all the anagrams in /usr/share/dict/words. The technique used was described by Jon Bentley in his magnificent book, Programming Pearls.

The Red Hat Package Manager (RPM) is an open packaging system, which runs on Red Hat Enterprise Linux as well as other Linux and UNIX systems like SuSE, Mandriva, Fedora and CentOS.

RPM is used to:

• Install new packages
• Uninstall existing packages
• Upgrade existing packages to the latest version
• Verify the integrity of existing packages

RPM also has powerful querying facilities. Those can be used, for example, for listing all installed packages, getting information on an installed package, identifying from which package a specific file is from, etc.

As RPM installs files obtained from the Internet, it also supports the use of cryptographic keys to authenticate packages.

Package Management Tool

RPM is a command line program and can be somewhat complex to use. For those who prefer graphical applications, there exists the a graphical package management tool.

YUM (Yellowdog Updater Modified)

YUM is now the standard tool for package management in Red Hat Enterprise Linux and derivatives. For most usual tasks, YUM should be used in preference to RPM as it supports new features that are now considered to be essential by most system administrators.

For example, YUM can install packages from various sources, called repositories. Those repositories can be on the Internet (e.g. an official Red Hat repository), on a LAN as well as local. Creating a local repository is a relatively straightforward task. This opens a lot of possibilities like downloading packages from a mirror, creating a repository of them on the LAN and configuring all computers in an organisation to use that rapidly-accessible repository.

Furthermore, YUM is better than RPM as it tracks (and resolves) dependencies among packages automatically. This makes installing complex packages trivial.

Package Management with APT

The Advanced Packaging Tool (APT) as used in Debian, Ubuntu and Linux Mint is the equivalent of RPM/YUM.

In general, system administrators prefer using the aptitude tool instead of the apt-get and apt-cache commands which are lower level tools with fewer capabilities.

aptitude can either be used as a command line tool or as a text application (see picture above). In both cases, aptitude can then:

• install a package with its dependencies
• remove a package while leaving its configuration files
• purge a package with its configuration files
• remove auto-installed packages which are no longer required
• display detailed information about packages
• search for packages which match a regular expression

aptitude is more advanced than YUM as it knows how to track dependencies both on installation and removals. YUM only resolves dependencies when installing packages.

Logging

EL6 Linux can be configured to log the various events occurring in files residing in /var/log. It is also possible to log to the console as well as to log to a remote log server.

Logging is done by the syslog demon which defines the following seven priorities: debug (less critical), info, notice, warning, err, crit, alert and emerg (the most critical). syslog is configured using the syslog.conf file.

The work to be done is to enable remote logging and make other computers log to it.

One important consequence of (heavy) logging is the need for log rotation. This makes sure that the log files remain at a manageable size and also takes care of archiving old log files. Log rotation can also be used to delete old log files which are not relevant anymore.

What is the relationship between cron and log rotation? How is log rotation configured?

Of course, the whole purpose of logging is to monitor the log files on a regular basis in order to detect unexpected events. Look at the various log files in /var/log and think about the different unexpected events that might appear in them and how those events might be detected.

At what frequency should log files be monitored? How?

In EL7 onwards, logging is managed by system and logs can be examined using journalctl.

Backup

A Linux system administrator should be able to:

• Plan a backup strategy
• Dump a raw device to a file or restore a raw device from a file (using dd for example)
• Perform partial and manual backups and restore the files (using tar for example)
• Synchronise two directories (using rsync for example)

There are various tools that are available for doing backups. Some are dated (e.g. dump/restore) and are being deprecated. It is important that the proper tools be used when needed.

System time

Finally, it is important that the time and date be properly set (EL6 and EL7) as essential services like cron and anacron depend on time being correct.

The work to be done is to ensure that the Linux server gets its time and date from a NTP (Network Time Protocol) server found on the Internet (i.e. become an NTP client). Setting up an NTP server is not required.

Geolocation and Google Maps are two important features of most Android devices.

Geolocation is a feature which Android applications can use to obtain the position of the device. Geolocation uses either the GPS, the GSM network or Wifi to determine (with different levels of accuracy) the latitude and longitude of the device. It can also determine the altitude, bearing as well as speed of the device if possible.

The application to develop should display all relevant geolocation information when the user clicks on Where am I?!? The location should be automatically updated when the device moves.

Essential Android classes for geolocation include LocationManager, LocationListener and Location.

Google Maps

The application will have to be enhanced to display a Google Map of the position of the device instead of just textual information. This Google Map will be updated whenever the device moves.

To display a map from Google Maps in Android 3.0 and up, it is important to use the Google Play Services.

• Ensure that you have installed the Google Play services application on your physical device.
• Make sure that the Google Play services is setup properly for the project in Android Studio. Google Play Services includes the Google Maps capability.
• Google Maps requires an API Key to work. Obtain the key corresponding to the debug certificate.
• Finally, the AndroidManifest of the project as well as the layout need to be modified accordingly.

Once this is done, the idea is to use classes provided by Google to control with the map shown.

In Android, an Intent is a message which allows you to request functionality from other Android components, whether those components are from the same application or from a different one. An intent can be used for instance to open an activity from the same app or to open an activity from another app.

SongExplorerForAll v7

In SongExplorerForAll v6, we created an about menu in the action bar. Here, clicking on the about menu displays a dialog which gives information about the app.

When creating an intent, we have to specify the type of action to perform. Here, we use ACTION_VIEW which is basically used when we need to display data to the user.

SongExplorerForAll v8

Another type of action is ACTION_SEND. This is used when we want the user to be able to share some data through another app. In SongExplorerForAll v8, when the user clicks on the share icon in the action bar, the latter will be able to share information about the Song Explorer For All app. The user is presented with a list of apps which can perform the sharing action (e.g. Gmail), from which he/she can choose.

There are two types of intents namely explicit intents and implicit intents. When using explicit intents, we have to explicitly specify the component to start by name. On the other side, when using implicit intents, we simply declare the action to perform. The Android system then finds appropriate component(s) to handle the intent. If there are several apps which can handle the intent, we can show a chooser dialog.

In order to provide a good user experience, one of the most important steps in designing and developing an Android application is to determine how the user will navigate the app. You should start by designing the screens hierarchy to clearly show the relationship among different screens. Then, decide on how to effectively traverse your content.

Android provides various navigation paradigms such as tabs, navigation drawers and up and back navigation. It is important to understand the best practices associated with those paradigms in order to provide a coherent navigation experience for users. Always remember that users expect similar components to behave in similar ways. Following best practices will avoid your users from getting lost and will make using your app easier.

SongExplorerForAll v4

One common pattern found in almost all Android applications is the navigation drawer. A navigation drawer is basically a panel which is revealed when the user swipes from the left edge of the screen (although a navigation drawer can also be found on the right). It generally displays a list containing the main top-level navigation options. In SongExplorerForAll v4, we add an empty navigation drawer to the app.

This is done by declaring a DrawerLayout as the root view layout of our Main Activity.

SongExplorerForAll v5

The SongExplorerForAll app displays a list of top artists worldwide. However, the Last FM Api also allows us to query a list of top artists based on top charts for a particular country (For example, the top chart artists in France might be different from the top chart artists in Spain). In SongExplorerForAll v5, we give the user the option to choose between either to display the top artists worldwide or to display the top artists based on top charts for Mauritius. The user chooses between those two top-level options via the navigation drawer. To achieve this we need to do the following:

• Create the appropriate url to query the top chart artists for a particular country
• Populate the list in the navigation drawer with the two values Top Artists Worldwide and Top Artists in Mauritius. Here, since there are only two items, it is not necessary to use a RecyclerView. A ListView with a simple ArrayAdapter is enough.
• On selecting one of the two items from the ListView, we need to get a reference to the ArtistListFragment being displayed and update it accordingly. Getting a reference to the fragment is done by first adding a tag to the fragment on creation and then using this tag to access it.

SongExplorerForAll v6

Another key component for navigating an Android app is the action bar. The action bar is located at the top of the screen and basically is used for

• Branding purposes (includes the application name, logo and branding color)
• Making important and commonly used actions prominent and easily accessible
• Ensuring consistent navigation together with tabs

In Android 5.0 Lollipop, the Toolbar was released which is basically a generalization of the action bar. While the action bar is part of the activity window, the Toolbar can be placed anywhere within a layout. A Toolbar can be set as the action bar using the setActionBar() method.

In SongExplorerForAll v6, we add a Toolbar to the Main Activity and set it as the action bar. We then add two menu elements, share and about. The share menu is displayed as a share icon in the Toolbar while the about menu is displayed when the user clicks on the more button (three vertical dots) in the action bar. When adding an icon, it is important to provide resources for the different density buckets (mdpi, hdpi, xhdpi, xxhdpi etc.) in order to ensure good graphical quality and performance on all screen densities.

Finally, we also add the drawer indicator in the action bar.

We have seen previously various best practices (such as using match_parent and wrap_content) for supporting different screen sizes. Android also allows us to provide several alternative layouts to target different screen configurations.

For example, let say we have an activity whose layout, found in the res/layout folder, is called activity_details.xml. This layout would be used for any device. However, Android allows us to create another folder, res/layout-large, in which we can have a different implementation of activity_details.xml. For example, in the activity_details.xml file found in res/layout, we can have an image whose width has been set to 100dp and in the activity_details.xml file found in res/layout-large, we can have an image whose width has been set to 200dp. In such a case, the activity_details.xml file found in res/layout-large would be selected on devices with screens classified as large.

Now, using the large size qualifier also has some limitations. Instead, we can use smallest-width qualifiers which allows us to specify specific screen sizes. For example, we can place the activity_details.xml file in the folder res/layout-sw600dp. Resources found in this folder would be used by devices whose smallest width is 600dp or more. Android also allows us to use orientation qualifiers such as res/layout-land where resources found in that folder would be used by devices in landscape mode.

Using the above mentioned qualifiers together with fragments, we can implement optimised and dynamic user interfaces which will adapt to different screen configurations quite easily.

SongExplorerForAll v3

In SongExplorerForAll v2, we used fragments to replace activities. In SongExplorerForAll v3, we create the new folder res/layout-sw600dp and add a different implementation of activity_main.xml in it to be used by screens whose smallest width is 600dp or more.

We also modify our MainActivity to display the appropriate fragments and perform appropriate tasks depending on whether the device has a larger screen. The result is that on smartphones, users get a list of artists and when selecting one of the artists get the artist’s details in another screen. On tablets, the list of artists and the artist’s details are displayed side by side.

Till now, we have seen how to use layouts and activities to implement different screens. Basically an activity takes several views, arrange them according to a specific layout and manage their behaviour as well as the application flow. Since so far we’ve developed applications for smartphones, activities were perfectly fine. However, once we want to optimise our application for tablets, activities can have various limitations.

When dealing with larger screens, designing user interfaces get a little bit more tricky. By using the same user interface for let say a 4 inch smartphone and a 10 inch tablet, a developer would not make optimum use of the space available on the screen.

If we have a look for instance at the SongExplorer v11 app on a 10 inch tablet, we can see that there’s a lot of space which is wasted on the right in the list of artists. One way to optimise the user interface for the tablet would be to use half of the screen for displaying the list of artists and use the other half for displaying the artist details such as the picture and tracks.

Unfortunately, since we have created the two screens using activities, this would require some complicated logic. Activities require full screen control when active and only one activity can be displayed on the screen at a time.

To address this issue, Fragments were introduced. Fragments not only can use a portion of the screen but also allow logic to be encapsulated into smaller building blocks and be reused. Fragments can be added statically to the UI but also dynamically to support different screen configurations.

SongExplorerForAll v1

In SongExplorerForAll v1, we implement the screen showing the list of artists and the screen showing the artist details using fragments instead of activities.

It is important to understand that fragments can only exist within a parent activity so we still need our MainActivity. We add the ArtistListFragment dynamically using a FragmentManager and FragmentTransaction.

You’ll notice here that on selecting a particular artist from the list, the artist details still open in a new activity since for now we haven’t told the application to replace the ArtistListFragment with the ArtistDetailsFragment. We are going to do that in the next version.

SongExplorerForAll v2

In order for fragments to be fully reusable, they should be built in such a way so as to be completely self-contained while at the same time being able to work with other components of the user interface. For instance, we should be able to reuse the ArtistListFragment in another application. However, while in the SongExplorerForAll v2 app the ArtistListFragment might need to communicate with the ArtistDetailsFragment, ArtistDetailsFragment might not necessarily be present in another app. This is why fragments should not communicate directly among each other. All communications among fragments should be performed through the parent activity.

In SongExplorerForAll v2, when an artist is selected from the list, it has to send this information to the ArtistDetailsFragment so that the latter updates itself accordingly. The information is first sent to the containing activity which then takes the appropriate action and passes the information to the ArtistDetailsFragment. This is done by creating an interface in the ArtistListFragment and implementing it in MainActivity.

While accessing data found on the device (data stored in an SQLite database, data stored on internal storage, data stored on external storage or data stored in Shared Preferences) might be fine for some apps, very often, an app might need to send and receive data from a remote server. The data can be either a plain text, XML, JSON or image.

When developing an Android application, we should avoid performing long running operations on the UI thread as this could result in a poor user experience. Sending a request and waiting for the response might take time and thus, as of Android 3.0, the Android system will crash if a network request is made within the UI thread. One way of dealing with network requests is therefore to use AsyncTask which basically allows you to implement multi threading and to perform background operations outside the UI thread. The results are then passed back to the UI thread. Using AsyncTask can however be tedious as you have to do a lot of things manually such as writing code to access the network.

Recently, Google has released a new framework called Volley which handles a lot of the tedious work for you. It also offers various advantages over AsyncTask such as caching, retry mechanism and various request types.

LastFMApp

Last FM, a popular music website, provides the Last.fm API which allows developers to build programs using Last FM data which includes details about artists and tracks among others. Once you create an API account and get an API key, the Last.fm API is free to use. The web service allows you to call methods that respond in REST style XML or JSON format.

Here, we use Volley and send a request to Last.fm API to get a list of top chart artists. The response obtained will be a JSON object which we display in a simple TextView.

When using Volley, network operations are managed by a RequestQueue to which you pass request objects.  A recommended practice is to create the RequestQueue as a singleton, thus ensuring that the RequestQueue lasts the lifetime of your app.

Also, in order for an application to be able to access the network, we should add the android.permission.INTERNET permission in our AndroidManifest.xml file.

LastFMAppReloaded

Now that we have a JSON object response, we need to parse it and retrieve data that is of interest to us. In LastFMAppReloaded, we retrieve the name of the first artist returned in the JSON response and display it in a TextView.

For this, we use the JSONObject and JSONArray classes and some of their methods such as getJSONObject() and getJSONArray().

LastFMAppReloadedwithImage

The Last FM Api also returns the urls for pictures of the different artists. In LastFMAppReloadedwithImage, we also retrieve the image for the first artist returned in the JSON response.

In Volley, this is pretty straight-forward using the ImageLoader class and the NetworkImageView component.

Song Explorer v9

Previously, we created the Song Explorer app which basically retrieved a list of artists from a database and displayed it in a listview (RecyclerView). Then on selecting one of the artists, we could get more information about that artist as well as a list of tracks for that particular artist.

Now that we know how to perform network requests with Volley, in Song Explorer 9, we’ll retrieve the list of artists from the Last FM web service instead of from the database.

You’ll notice that selecting a particular artist from the list returns an error. This is because the artist id we used in the database is different from the artist id used by the Last FM api. We’ll fix that in Song Explorer v10.

Song Explorer v10

The Last FM Api also has a method which allows us to retrieve a list of top tracks for a particular artist. In Song Explorer v10, when selecting a particular artist from the list, we are redirected to another activity just like in the previous versions of Song Explorer but this time, we display the artist name, the artist image and a list of top tracks for that artist.

Song Explorer v11

Finally, just like we did for the ForexWithStyle app, we’ll modify the style and add some colors to the Song Explorer app.

Data-centric applications normally have to display a lot of information and Android has two important widgets which can be used to display a lot of information in a small space:

• Spinner: We have used a spinner when developing the Forex app previously. We have seen how it needs an adapter to display its data.
• ListView/RecyclerView: This is a scrollable list of items which can be clicked on. ListViews/RecyclerViews also require an adapter to work.

In the previous lesson on databases we developed the Song Explorer app which basically retrieves a list of artists and tracks from a database and displays it on the screen using a TextView. However, using a TextView for this purpose has some limitations. For instance, we cannot select one specific item and click on it to get more details. The best way to display a list of items in Android is to use a ListView/RecyclerView.

ListView vs RecyclerView

When using lists to display data, you need to get references to each view component for each item in the list. If your list has only a few items that’s fine but if your list has let say hundred items, repeatedly calling findViewById() can slow down performance and as a result the scrolling experience won’t be smooth. To address the issue, it is recommended to use a pattern called the ViewHolder pattern which basically recycles views when they go out of the screen when you scroll. So instead of creating new view components for each item in the list, we reuse those which are not displayed on the screen.

Previously, one could make ListView scrolling smooth by implementing the ViewHolder pattern in the corresponding ListView adapter. However, this was an optional thing. With the advent of Android Lollipop, RecyclerView was introduced. RecyclerView is a much more powerful, flexible and a major enhancement over ListView. For instance, RecyclerView makes the use of ViewHolder compulsory using the RecyclerView.ViewHolder class. It also has additional functionalities for item animation and item decoration among others.

Song Explorer v4

Here, we’ll develop version 4 of the Song Explorer app. Instead of displaying the list of artists in a TextView, we’ll display it using a nice RecyclerView. For that, we’ll also have to create ArtistAdapter class which will extend RecyclerView.Adapter.

Song Explorer v5

Till now, we have only displayed the artist name. However, in the database, we also have the artist id and the artist bio. In Android, it is possible to customise lists to display more than one piece of information for each item in the list. For example, for one item, we can display a thumbnail image to the left and other pieces of information to the right of the image. In version 5 of the app, we learn the basics of creating custom lists by displaying the artist id together with the artist name.

Song Explorer v6

In version 5 of the Song Explorer app, we have a nice RecyclerView displaying a list of artists which includes the artist’s id and the artist’s name. As mentioned previously, one interesting feature of using ListViews/RecyclerViews is that we can click on a specific item in the list. Till now, when clicking on a specific item, nothing happens. In version 6, when clicking on a specific item, the name of the artist is displayed in a Toast.

Song Explorer v7

As you are probably guessing, displaying the name of the artist in a Toast is not very useful. In version 7, when clicking on an item in the list, we display more details about that particular artist (i.e. artist name, bio and tracks)on a different screen. To achieve this, the following needs to be done:

• Create a new activity, ArtistDetailsActivity, and add it to the AndroidManifest.xml file
• Update the onClick() in ArtistAdapter to start the new activity and pass in the artist id
• Get the artist id in the new activity (ArtistDetailsActivity) and retrieve the corresponding artist name and bio from the database and display those in two respective TextViews
• Retrieve the list of tracks for that particular artist from the database
• Create a RecyclerView and a corresponding TrackAdapter to display the list of tracks for the artist

Song Explorer v8

The last enhancement to bring to the application is to allow the user to specify whether the list of artists should be sorted by the artist id or by the artist name. The next time the application starts, the list should be displayed in the chosen order. To do that, a preference screen should be created using the XML editor, then a class (SettingsActivity) derived from PreferenceActivity should be created and the onCreate method overriden. This new activity should be manually added to AndroidManifest.xml.

When this activity is launched (using, for example, startActivity), the Android framework will automatically store the preferences.

Here, the preference screen will be launched when the user select settings in the menu which can be accessed through the bar at the top (also known as the action bar). To do this, startActivity() should be called accordingly in the onOptionsItemSelected() method in MainActivity.

You will notice that the preference screen doesn’t have the action bar at the top. This is because unlike MainActivity, SettingsActivity extends PreferenceActivity instead of AppCompatActivity. We will see later on what AppCompatActivity is all about and how to add an action bar to SettingsActivity too.

Finally, the application will have to be updated to display the list of artists in the appropriate order based on the user’s selection. The way to do that is to get a SharedPreference object from the PreferenceManager, retrieve the corresponding setting value and use the appropriate SQL statement when querying the database.