rxjava and a synchronous main thread

not too long ago, someone pointed me to this blog post about keeping your main thread synchronous. it referenced Ray Ryan’s excellent talk about the matter. the talk and blog post lead me to a set of investigations, which prompted me to write this blog post.

quick note: why keep the main thread synchronous

if you are curious as to the kinds of problems that can occur if the main thread isn’t synchronous (or why it’s not synchronous when you use observeOn), see this post about the main thread for a good explanation.

the summary is that everytime you use handler.post(), you post something to be run later, and you don’t have any guarantees as to when it will be run. a common case is when you schedule something to update the ui, but before it actually runs, an onDestroy comes in, causing the code to update the ui after the destruction of the activity.

in rx, specifically, observeOn(AndroidSchedulers.mainThread()) causes a handler.sendMessageDelayed (see LooperScheduler.java), which could cause code to run at a point after we are thought to have unsubscribed, thus causing issues.

basic rules of rxjava threading

in many of the talks about rxjava1, we find a set of repeated rules about rxjava and threading:

  • rxjava is single threaded by default unless you explicitly ask it otherwise
  • subscribeOn only affects upstream
  • only the subscribeOn closest to the source matters
  • observeOn only affects downstream

while these are all true, there are a few minor points that were not immediately obvious to me early on about the first three, so i would like to elaborate a bit on those.

rxjava is single threaded by default

as long as you do not use observeOn, subscribeOn, or an operator that runs on a particular scheduler (ex timer), the callback will be receieved on the thread subscribe happened on.

subscribeOn only affects upstream

one subtle point - consider:

Observable.just(1, 2, 3)
    .subscribeOn(Schedulers.io())
    .subscribe(integer -> {
      Log.d(TAG, "got value on " + Thread.currentThread().getName());
    });

despite subscribeOn only affecting upstream, this will always print the result on an io thread, irrespective of the thread on which we called this code. this is because subscribeOn subscribes to the observable on the thread passed in, which means that onNext will be called on that particular thread.

only the subscribeOn closest to the source matters

consider this code:

Observable.just(1, 2, 3)
    .doOnSubscribe(() -> 
        Log.d(TAG, "subscribe to just on " +
            Thread.currentThread().getName()))
    .subscribeOn(Schedulers.io())
    .filter(integer -> integer % 2 == 0)
    .doOnSubscribe(() ->
        Log.d(TAG, "subscribe to filter on " +
            Thread.currentThread().getName()))
    .subscribeOn(Schedulers.computation())
    .subscribe(integer -> {
      Log.d(TAG, "got value on " + Thread.currentThread().getName());
    });

running this example results in:

D/are: subscribe to filter on RxComputationScheduler-1
D/are: subscribe to just on RxIoScheduler-2
D/are: got value on RxIoScheduler-2

if, however, we changed doOnSubscribe with doOnNext in the code block above, we’d instead get:

D/are: onNext from just with RxIoScheduler-2
D/are: onNext from just with RxIoScheduler-2
D/are: onNext from filter with RxIoScheduler-2
D/are: got value on RxIoScheduler-2
D/are: onNext from just with RxIoScheduler-2

the caveat here is that the subscribeOn closest to the source is the one that determines which thread onNext will get called on (but subscriptions still happen on the thread specified by subscribeOn).

the reason for this is that each subscribeOn subscribes to the upstream observable on that particular thread.

let’s take an example - given:

Observable.just(1,2,3)
          .subscribeOn(Schedulers.io())
          .subscribeOn(Schedulers.computation())
          .subscribe();

let’s break down what happens:

diagram

  • Observable.just(1,2,3) internally calls Observable.create(new OnSubscribeFromArray()), so we have an Observable which will call OnSubscribeFromArray on subscribe.

  • we then call subscribeOn(Schedulers.io()), which calls Observable.create(new OperatorSubscribeOn(this, scheduler)), where this is the Observable from above, and scheduler is io. In other words, we now have an Observable, which has an onSubscribe that will subscribe to the Observable from the step above on the io thread.

  • we then call subscribeOn(Schedulers.computation()), which calls Observable.create(new OperatorSubscribeOn(this, scheduler)), where this is the Observable from above, and scheduler is computation.

  • finally, we call subscribe, which calls Observable.subscribe, which calls the method from OperatorSubscribeOn from point 3 - this subscribes to the Observable from point 2 on the computation thread. Ultimately, this causes the OperatorSubscribeOn from point 2 to be called, which then calls subscribe to the Observable from point 1 on the io thread. OnSubscribeFromArray produces values on the same thread, thus causing all the items to be emitted on the io thread.

see the source for subscribeOn for more details.

running things in parallel with rxjava

in order to run things in parallel, we use flatMap or concatMap, with multiple observers that can then subscribeOn whatever scheduler they want to. the difference between concatMap and flatMap is that flatMap can emit items out of order, whereas concatMap will always emit items in order.

so what does this do? flatMap is essentially a merge, which “combines multiple Observables into one by merging their emissions”2. note that the observable contract stipulates that “Observables must issue notifications to observers serially (not in parallel).” this means that onNext will not be called concurrently, and part of merge’s job is to make sure that onNext is only called by one thread at a time.

for more on this, see Thomas Nield’s article about achieving parallelization, and also, see David Karnok’s article about FlatMap.

special thanks to Michael Evans for proofreading this.


  1. both of the aforementioned talks are definitely worth watching if you haven’t already seen them! [return]
  2. quote from rx merge docs [return]

rtl on Android

i gave my first talk at droidcon sf this year. the talk was about RTL on Android. check it out if you haven’t already (the slides are here).

on LinearLayout measures

seeing that LinearLayout is one of the most often used ViewGroup types in Android development, i wanted to write a little bit about how LinearLayout measures its children (this was also inspired by Sriram’s post about Custom ViewGroups).

tldr; (key takeaways that this post will discuss):

  1. a (non-nested) LinearLayout will measure each non-hidden child either 1x, 2x, or 3x (gone views aren’t measured).
  2. whenever possible, set android:baselineAligned to false, especially when using weights.
  3. in a horizontal LinearLayout with wrap_content height, avoid setting any of the children’s heights to match_parent. conversely, in a vertical LinearLayout with wrap_content width, avoid setting any of the children’s widths to match_parent.

LinearLayout and child measurement

a (non-nested) LinearLayout will end up measuring each non-hidden view either 1x, 2x, or 3x. here are the rules:

  1. a View with android:visibility="gone" won’t be measured.
  2. a View will default to being measured once, unless more measurements are necessary (see rules 3 and 4).
  3. when the non-android:orientation direction of the LinearLayout is wrap_content and its child’s is match_parent, the child will be measured an extra time.
    • in a horizontal LinearLayout with wrap_content height, a child with match_parent height will be measured an extra time.
    • in a vertical LinearLayout with wrap_content width, a child with match_parent width will be measured an extra time.
  4. a child with a non-zero android:layout_weight will be measured an extra time if:
    • in a horizontal LinearLayout with non-wrap_content width, android:layout_width is not 0dp, (or is 0dp, but the LinearLayout doesn’t have android:baselineAligned="false" set).
    • in a vertical LinearLayout with non-wrap_content height, if android:layout_height is not 0dp.

the first two rules are fairly straight forward - for optimization purposes, a view that is set to gone will not be measured. secondly, every view must be measured at least once, so the default is that each view will be measured once, unless something causes it to have to measure more.

the width/height mismatch measure

consider a horizontal LinearLayout with two TextView children.

if the height of this LinearLayout is match_parent or a fixed value (ex 32dp), then measuring the height of the children is easy:

  • if the child has a fixed height, use that
  • if the child’s height is match_parent, then pass in the LinearLayout’s height with a MeasureSpec mode of EXACTLY.
  • if the child’s height is wrap_content, then pass in the LinearLayout’s height with a MeasureSpec mode of AT_MOST.

but what if the LinearLayout’s height is wrap_content? in this case, measuring the child is sometimes easy:

  • if the child has a fixed height, use that.
  • if the child has a height of wrap_content, pass a height MeasureSpec with a mode of AT_MOST and the available space to the LinearLayout.

if, however, the child has a height of match_parent, what we’re really saying is, “i want this view to have the same height as the LinearLayout.”

so what is the height of the LinearLayout? in the case where the LinearLayout’s height is set to wrap_content, the height is not known until each child has been measured (the height MeasureSpec received in onMeasure by the LinearLayout itself will be AT_MOST with however much space can be taken up by it).

consequently, children with a match_parent height also have to wait until the LinearLayout’s height is determined, and then measured again in accordance to that height.

note that the same is true for vertical LinearLayouts when they have wrap_content widths and children with match_parent widths.

(note that in LinearLayout.java, you’ll see forceUniformHeight and forceUniformWidth in measureHorizontal and measureVertical, respectively, being called at the end of the measure cycle when they need to be).

example

here’s an example that shows this effect - compare:

the right square in the first image is measured twice, because its height is set to match_parent when the LinearLayout’s height is set to wrap_content. if these match (ex both are wrap_content, both are match_parent, or one of them is fixed), we eliminate this problem1.

note that sometimes, you really have to use match_parent - for example, if the right side had a background and we wanted it to be exactly the same height as the left side, we have no choice but to use match_parent for the height (unless we can specify a fixed height or change the parent, etc).

weights

setting a non-zero android:layout_weight may cause a View to be measured an extra time.

first, let’s consider when a non-zero android:layout_weight does not cause an extra measure pass -

for a vertical LinearLayout:

  • when the LinearLayout has a layout_height of wrap_content, there is no extra measure (this makes sense, since there is no “extra space” to divvy up amongst the views at the end, since we’re telling the LinearLayout to only be as large as its content).
  • when the layout_height of the child is 0dp.

for a horizontal LinearLayout:

  • when the LinearLayout has a layout_width of wrap_content, there is no extra measure.
  • when the layout_width of the child is 0dp, and the LinearLayout itself has android:baselineAligned="false" set.

in all other cases, the view with non-zero layout_weight will be measured an extra time.

android:baselineAligned

the documentation for android:baselineAligned says:

When set to false, prevents the layout from aligning its children’s baselines. This attribute is particularly useful when the children use different values for gravity. The default value is true.

this is generally useful for aligning text with the same gravity. consider:

and compare it to:

here’s a more visual representation:

so when can you turn it off?

  1. if the children of the LinearLayout don’t need to be baseline aligned (i.e. are not TextViews or ImageViews with setBaseline or setBaselineAlignedBottom set, etc).
  2. if the children are TextViews but all have different alignments anyway.
  3. if it doesn’t matter (for example in the above screenshots, because both sides use English text with the same text size, the images are identical - if they were different languages, or if the text size of one was different than the other, the two would definitely be different).

here’s a visual representation of how baselineAlignment can make a difference:

why does this matter?

why write a blog post about this, and why does it matter? there are 2 main reasons - first, given the popularity of LinearLayout within most apps, its important to know how it works to have great performing apps.

secondly, measures can be expensive - and avoiding extra work is always great! this is especially relevant when you consider nested LinearLayouts - let’s suppose that a nested LinearLayout will be measured 3 times, and has children that it would ordinarily measure 3x - those children now get measured 72 times!


  1. note that this is only true for a non-nested LinearLayout - if instead, the aforementioned layout is the child of a vertical LinearLayout with a height of wrap_content, the TextView with match_parent height will still get measured twice, because the extra measure at the end happens when a child is match_parent and the parent’s height MeasureSpec is not EXACTLY (in this case, its AT_MOST due to wrap_content). [return]
  2. in theory, they’d get measured 9 times, but the variation in the specs that are passed in to the nested LinearLayout let some of the views skip extra measures, especially in the first 2 measure passes. [return]

intellij structural replace

intellij’s structural search and replace is an amazing feature that can save a lot of time. the value of this feature first hit home in the excellent Android Studio for Experts talk during Android Dev Summit 2015 (excellent video if you haven’t already seen it, btw!)

today, i was trying to replace my usages of android.util.Log.* with Timber. to use Timber, we’d need to manually replace Log.d(TAG, "message"); with Timber.d("message");, and the same for Log.e, Log.w, and so on. there’s also a version of Log.* that takes in an exception as a paramter that we have to keep in mind as well.

sounds like a perfect job for structural search - we bring it up by using ⌘ + shift + A to search for actions. type Structurally, and choose Replace Structurally.

we’ll get a dialog that we’ll fill out like this:

so to do this, set the search template to:

android.util.Log.$something$($tag$, $params$);

set the replacement template to:

timber.log.Timber.$something$($params$);

note that the fully qualified domain name in the replacement ensures that the import is handled for you, as long as shorten fully qualified names is checked.

finally, choose Edit variables, choose params on the left, and click the unlimited box by maximum count. this is so that we match both Log.d(TAG, "message"); and Log.d(TAG, "message", exception);.

some notes about the options:

  • shorten fully qualified names replaces things like android.util.Log.d with Log.d.
  • reformat according to style will fix the indentation if it’s wrong.
  • use static import if possible will prefer import static - i’ve unchecked this because i’ve found that it did this for many things that i didn’t want it done for.

and there you have it. as always, source control is your friend, so make sure to do this on a branch where you can validate the changes and easily roll back if things aren’t quite right.

quran android no longer open source

update - as of 12/31/2015, i’ve re-open sourced the app after the discussion here.

today is a very sad day for me, since i have finally decided to close quran android’s source code once and for all. throughout the past few years, i’ve come across several examples of people republishing the app with the intention of making money (placing ads on the index page, or, worse yet, on the top and bottom of each quran page while reading). recently, someone brought to my attention an example of someone repackaging the apk and shipping it with malware.

for example, fabric shows me this crash:

this issue is happening on the current version in the play store, which was pushed on 5/31/2015. looking at the graph, you can see that until mid July, there were no crashes (because this sdk doesn’t exist in our app). after mid-July, someone pushed this (modified) apk somewhere, people started downloading it, and it started reporting crashes to me.

a similar thing happened some time back, but with the person pushing their own version:

a few years ago, i closed the source to quran android for similar reasons, only to end up bringing it back after a few months, figuring the benefit of it being open outweighs the harm. unfortunately, this time, the decision is not meant to be reversed insha’Allah.

i’ve come to the difficult realization that apps are not meant to be open sourced, because, when you open source it, there will always be people who try to profit off of your work (or destroy its credibility, etc).

quran will, insha’Allah, continue to be developed, but it won’t be open source anymore. the existing github project will be used to track issues. while i understand that removing the code will not completely stop these things from happening (apks are very easy to reverse engineer, and people with a will to do something will find a way), at the very least, i can stop making it super easy for people.

update - this seems to be a trojan that affects multiple apps downloaded from unofficial app stores.

migrating to hugo from hexo

i haven’t written in a long time. it’s not that i have nothing to write about (i actually have a set of post ideas written somewhere), but more of just not having the time for writing.

just yesterday, after seeing a fellow developer’s new blog based on jekyll, i decided to search for a nicer theme for this blog. in the process, i came across hugo, a blogging engine written using go. the two things that interested me the most about it was the fact that it was supposed to be super fast (whereas regenerating files for hexo still took a bit of time), that it was written in go (which i hope to learn one day), and that i found a theme i liked for it. the fact that chrome on the latest el capitan beta doesn’t load my site at all was yet another reason to push me to go for it.

the process didn’t end up being too bad, and i’ll document some interesting tidbits i found here in case someone else goes through the migration in the future.

fixing the metadata.

there are a few important pieces required to fix the metadata - first, ensuring all lines start with ---. for me, any files i generated on hexo didn’t have --- on the top of them. secondly, fixing the dates. hexo dates were in a different format than the iso8601 format dates. furthermore, some of the entries i had wrapped their dates with single quotes.

i found something here to do this, and modified it to fix the single quote issue as well:

# ensure dates don't start with single quotes
for file in *; do awk '{
if ($1 == "date:") {
  gsub("\047", "", $0); print;
} else {
  print $0;
}
}' "$file" > temp.md && mv temp.md $file ; done

# fix the dates and add the three dashes as the first line
for file in *; do awk '{
  if (NR == 1) { print "---"; }
  if ($1 == "date:") {
    printf("%s %sT%s-05:00\n", $1, $2, $3);
  } else {
    print $0;
  }
}' "$file" > temp.md && mv temp.md $file ; done

# wrap dates with quotes that aren't wrapped in quotes
for file in *; do awk '{
  if ($1 == "date:") {
    if ($2 ~ /^"/) {
      print $0;
    } else {
      printf("%s \"%s\"\n", $1, $2);
    }
  } else { print $0; }
}' "$file" > temp.md && mv temp.md $file; done

one last thing was that some of the entries i had were missing the seconds field, and hugo complained about being unable to parse their times. since they were only a handfull of entries, i went ahead and fixed them manually.

fixing code highlighting

i used something like this to convert the codeblock style to the pygments driven code highlighting:

for file in *.md; do awk '{
if ($2 == "codeblock") { gsub("lang:", "", $(NF-1));
  printf("%s< highlight %s >}}\n", "{{", $(NF-1));
} else if ($2 == "endcodeblock") {
  printf("%s< /highlight >}}\n", "{{");
} else { print $0; }
}' "$file" > temp.md && mv temp.md $file ; done

as a recommendation, if trying to use the hyde theme, set pygmentsUseClasses to true in the configuration file.

after doing this, i realized that i wanted to use hyde-x, which by default uses highlight.js. i made this work (after running the above) by running:

find . -exec sed -i '.bak' 's/{{< highlight \(.*\) >}}/```\1/' {} \;
find . -exec sed -i '.bak' 's/{{< \/highlight >}}/```/' {} \;

fixing images

hugo supports shortcodes as a means for extending markdown. using a combination of the migrate to hugo from jekyll article, the shortcodes documentation, and migrating to hugo from octopress, i came up with a similar approach for fixing images that had a width and height in them.

find . -exec sed -i '.bak' 's/{% img \([^ ]*\) \([^ ]*\) \([^ ]*\) %}/{{< img src="\1" width="\2" height="\3" >}}/' {} \;

this was combined with adding an img shortcode, which looked like this:

<img src="{{ .Get "src" }}"
{{ if .Get "width" }} width="{{.Get "width" }}" {{ end }}
{{ if .Get "height" }} height="{{.Get "height" }}" {{ end }} >

in addition, i modified the css to remove display:block from poole.css under images to allow having two images side by side.

one other minor thing i found was that layout: post, which was either an artifact of hexo or octopress, was breaking disqus rendering. i fixed this by doing this:

for file in *.md; do
   grep -v "layout: post" $file > temp.md
   mv temp.md $file
done

a little bit of extra tweaking to the settings and layouts and i was good to go! hopefully, this will push me to write posts a little bit more frequently :)

mouse scrolling with RecyclerView

i’ve recently began using RecyclerView in some of my projects, and one thing that stood out as being broken was that mouse scrolling no longer worked. while this is perhaps only a theoretical issue, it is fairly vexing when developing on an emulator, especially while dealing with long lists of data. i’ve opened a bug about it.

this is solved by using onGenericMotionEvent. as per the documentation, “Generic motion events describe joystick movements, mouse hovers, track pad touches, scroll wheel movements and other input events.” for our RecyclerView, we basically have two options - one is to use setOnGenericMotionListener, and the other is to have our own subclass of RecyclerView to handle this.

below, you will see the subclass solution (which can be easily adopted for use with setOnGenericMotionListener as well). the code here is adopted from AbsListView.

public class CustomRecyclerView extends RecyclerView {
  private Context mContext;
  private float mVerticalScrollFactor;

  // constructors omitted for brevity

  private float getVerticalScrollFactor() {
    if (mVerticalScrollFactor == 0) {
      TypedValue outValue = new TypedValue();
      if (!mContext.getTheme().resolveAttribute(
          R.attr.listPreferredItemHeight, outValue, true)) {
        throw new IllegalStateException(
            "Expected theme to define listPreferredItemHeight.");
      }
      mVerticalScrollFactor = outValue.getDimension(
          mContext.getResources().getDisplayMetrics());
    }
    return mVerticalScrollFactor;
  }

  @TargetApi(Build.VERSION_CODES.HONEYCOMB_MR1)
  @Override
  public boolean onGenericMotionEvent(MotionEvent event) {
    if ((event.getSource() & InputDevice.SOURCE_CLASS_POINTER) != 0) {
      if (event.getAction() == MotionEvent.ACTION_SCROLL &&
          getScrollState() == SCROLL_STATE_IDLE) {
        final float vscroll = event.getAxisValue(MotionEvent.AXIS_VSCROLL);
        if (vscroll != 0) {
          final int delta = -1 * (int) (vscroll * getVerticalScrollFactor());
          if (ViewCompat.canScrollVertically(this, delta > 0 ? 1 : -1)) {
            scrollBy(0, delta);
            return true;
          }
        }
      }
    }
    return super.onGenericMotionEvent(event);
  }
}

haramain for android

one of my favorite sites on the web is haramain recordings. haramain recordings posts the latest prayers from masjid al haram and masjid al nabawi on a daily basis (every fajr, maghrib, and isha salah, along with jum3a khutbas, salat al tarawee7, and more).

because i found myself visiting haramain recordings very frequently, i started thinking, “it would be really great if i could access this on my phone…” and so, al7amdulillah, haramain for android was built.

haramain is an unofficial client for haramain recordings. it lets you stream the latest prayers from both masajid, and even lets you filter by sheikh. it was designed in accordance to material design, so it should look great on android L insha’Allah. it’s also free with no ads.

it’s quite light weight at the moment, and there are a lot of things that are missing, but insha’Allah as time goes on, we hope to keep updating it and make it more useful to people.

the app’s account is on Twitter at @haramainapp, and you can download it from google play.

ripples with probe

today, i was having a discussion with Mostafa Gazar about android animations and his Widgets library, and Lucas Rocha’s probe project came to mind.

i decided to experiment and see if i could use probe to automatically add a ripple effect to a set of views without having to touch existing code. turns out it’s fairly easy to do! i did two separate implementations.

the first image is using Markus Hi’s android-ui library. the second image is using Mostafa’s Widgets.

the code using Mostafa’s Widgets is a lot simpler (due to RippleView doing most of the heavy lifting) - you can see it here. the code for wrapping android-ui is a bit more complicated (and mimics what Mostafa does in RippleView) and can be found here.

the general idea for both probe implementations is the same - we take any given view, and replace it with a FrameLayout containing the view, and an underlying “ripple view.”

now to apply either of these methods, we can do something like this:

public class RippleActivity extends Activity {

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    Probe.deploy(this, new RevealInterceptor(this),
        new Filter.ParentId(R.id.ripple_main));
    // can also use RippleInterceptor instead
    super.onCreate(savedInstanceState);
    setContentView(R.layout.ripples);
  }
}
<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/ripple_main"
    android:orientation="vertical"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <View android:id="@+id/button"
        android:layout_width="128dp"
        android:layout_height="128dp"
        android:background="#aa00aaff"
        android:layout_gravity="center" />

    <View android:layout_gravity="bottom"
        android:layout_width="50dp"
        android:layout_height="50dp"
        android:background="#aaff" />

</FrameLayout>

one interesting point - if you remove the filter and just let probe work on any view (both interceptors enforce non-ViewGroups only), the ripple effect ends up working on all views, including views in the ActionBar as well (which is really neat!)

saudi matches 1.0.1

not too long back, i posted about euro matches. earlier this year, we also pushed saudi matches. it includes a new ui, new features, and so on. saudi matches was done in conjunction with Khalid Sudairy, Bandar Raffah, and Taylor Ling.

you can download it here.