Web scraping in sketchware| Sketchware tutorial

Hi friends.

I welcome you to another sketchware tutorial. In this tutorial I Wil show you how you can convert your website or blogger blog into a good looking android app. We will be using web scraping in sketchware to parse the html. But before that, Please note this article Is going to be a little bit longer. As because I will let you create the app in simple steps and as well as explain each steps.

Please note that as every website need to analyse separately and each website need different codes for web scraping. I created this code for the contempo theme in blogger.

if you have your blog on blogger and using contempo theme.

Only then it will work Otherwise it won work.

Important: this project a library called jsoup. Jsoup library is not directly available to the sketchware app. We need to download the sketch library tool from play store. Inject jsoup library in to your project. Without jsoup library your app will through errors.

Let’s start this tutorial.

Step No. 1

In this step we will create the required variables. First Create an new project in sketchware.

  1. Add 4 String variable named url, img, href, title
  2. Create a map variable called map
  3. Create a List map variable named Lmap.

And that’s all in the variable part. We will use the url String to store your url. The img, href and title will store the image, post link and post title respectively.

Step No. 2

Now in this step we will create the required components

We only need the Intent component

create a Intent and name this as link.

Step No. 3

We will create views in this step.

  1. We will create a custom view named cust for our listview
  2. And a view called web.xml we will load the post in this view

Step No. 4

Now in your main.xml add a listview a linear(H) and a progressbar

Put the listview at the top. Then put a linear(H), and inside it put a progressbar. Set cust as your custom view for listview.

Now set the height and width of your linear(H) as match parent and set it’s gravity to center horizontal and center vertical.

For the progressbar set it height and width 80px and I set indeterminate true. The final view should look like this

Step No. 5

Now go to the custom view cust. And put a ImageView and two textviews

The ImageView Wil be the blog post image and the two titles will be the post title and post link. I have designed them as below. You can chose your own

Step No. 6

In your mainActivity go to on create and set blocks as below image

Set the String url to your own url. In my case it was https://sketchwarecodes.com, you need to put your own url in this block.

Do rest of the things exactly what I did. Else the app may show error.

The codes for the add source directly blocks are

ASD 1

foo();
}
private class AsyncTaskEx extends AsyncTask {
int linkNo =0;
@Override
protected Void doInBackground(Void… arg0) {
try{
org.jsoup.nodes.Document doc = org.jsoup.Jsoup.connect(url).get();
org.jsoup.nodes.Element body = doc.body();
org.jsoup.select.Elements links = body.select("article[class=post-outer-container]");
for (org.jsoup.nodes.Element link : links) { try{
org.jsoup.nodes.Element linkdata = link.select("a").last();
org.jsoup.nodes.Element imgurl =
link.select("img").first();
href = linkdata.attr("href");
img = imgurl.attr("src");
title = linkdata.attr("title");

ASD 2

}
catch(Exception e){} } } catch(Exception e){} return null;
}
@Override protected void onPostExecute(Void result) {

And the ASD 3

return ; } } private void foo(){ new AsyncTaskEx().execute();
try{
} catch(Exception e){}

Step No. 7

Go to the view event and add two event called listview onBindCustomView and listview onItemClicked

Go to onBindCustomView and set blocks as the below image.

Now go to listview onItemClicked and set blocks as below

Step No. 8

Go to your web.xml and drop a webview in it.

And in your webActivity.java go to onCreate event and set blocks like below

And run your project. Your web scraping app in sketchware is ready. My app looks something like this after it load all my articles

Hope your app runs smoothly. If you face any problem. Please comment me below. I will try to solve your problem.

Here is a link of my project’s source code. you can download it. Run this project in your android studio and see the result.

You can learn more about jsoup library and web scraping from this link

You can also check out our other posts from here

Thanks for reading

This Post Has 2 Comments

  1. DNA Mobile

    Hey so I followed everything and the app compiles but doesn’t load information to the listview. I tested with your website and it’s not loading like it should.

    1. Kaushik

      Hi there as I already mentioned in my post that every site needs different codes for websraping. And the code I mentioned in my post works on a blogger site with contempo theme. So in your case if you try with sketchwarecodes.com website then your app should not load any thing. You can try loading this website allsketchwarecodes.blogspot.com
      It should work fine.

Leave a Reply