2. Tutorials
  3. Retrofit 2 — Crawl HTML Responses with jspoon (Wikipedia Example)

Retrofit 2 — Crawl HTML Responses with jspoon (Wikipedia Example)

Retrofit is mainly used to interact with well-defined REST APIs. Nevertheless, you can also use Retrofit to crawl HTML sources of websites. In this tutorial, you'll explore how to utilize jspoon and Retrofit to automatically extract content from Wikipedia.

In a practical example you'll understand the basic functionality of jspoon and create a small Android app to extract the important bits of a Wikipedia article. Finally, you'll automate this extraction process with a Retrofit converter.

Retrofit Series Overview

  1. Getting Started and Creating an Android Client
  2. Basics of API Description
  3. Creating a Sustainable Android Client
  4. URL Handling, Resolution and Parsing
  5. How to Change API Base Url at Runtime
  6. Multiple Server Environments (Develop, Staging, Production)
  7. Share OkHttp Client and Converters between Retrofit Instances
  8. Upgrade Guide from 1.9
  9. Beyond Android: Retrofit for Java Projects
  10. How to use OkHttp 3 with Retrofit 1
  1. Synchronous and Asynchronous Requests
  2. Send Objects in Request Body
  3. Add Custom Request Header
  4. Manage Request Headers in OkHttp Interceptor
  5. Dynamic Request Headers with @HeaderMap
  6. Multiple Query Parameters of Same Name
  7. Optional Query Parameters
  8. Send Data Form-Urlencoded
  9. Send Data Form-Urlencoded Using FieldMap
  10. How to Add Query Parameters to Every Request
  11. Add Multiple Query Parameter With QueryMap
  12. How to Use Dynamic Urls for Requests
  13. Constant, Default and Logic Values for POST and PUT Requests
  14. Cancel Requests
  15. Reuse and Analyze Requests
  16. Optional Path Parameters
  17. How to Send Plain Text Request Body
  18. Customize Network Timeouts
  19. How to Trust Unsafe SSL certificates (Self-signed, Expired)
  20. Dynamic Endpoint-Dependent Interceptor Actions
  21. How to Update Objects on the Server (PUT vs. PATCH)
  22. How to Delete Objects on the Server
  1. Introduction to (Multiple) Converters
  2. Adding & Customizing the Gson Converter
  3. Implementing Custom Converters
  4. How to Integrate XML Converter
  5. Access Mapped Objects and Raw Response Payload
  6. Supporting JSON and XML Responses Concurrently
  7. Handling of Empty Server Responses with Custom Converter
  8. Send JSON Requests and Receive XML Responses (or vice versa)
  9. Unwrapping Envelope Responses with Custom Converter
  10. Wrapping Requests in Envelope with Custom Converter
  11. Define a Custom Response Converter
  1. Simple Error Handling
  2. Error Handling for Synchronous Requests
  3. Catch Server Errors Globally with Response Interceptor
  4. How to Detect Network and Conversion Errors in onFailure
  1. Log Requests and Responses
  2. Enable Logging for Development Builds Only
  3. Log Network Traffic with Stetho and Chrome Developer Tools
  4. Using the Log Level to Debug Requests
  5. Analyze Network Traffic with Android Studio Profiler
  6. Debug and Compare Requests with RequestBin
  1. Introduction to Call Adapters
  2. Custom Call Adapter to Separate OnResponse Callback
  3. How to Integrate RxJava 1.x Call Adapter
  4. How to Integrate RxJava 2.x Call Adapter
  5. How to Integrate Guava Call Adapter
  6. Custom Call Adapter to Separate Network and Gson Errors
  1. Pagination Using Query Parameter
  2. Pagination Using Link Header and Dynamic Urls (Like GitHub)
  3. Pagination Using Range Header Fields (Like Heroku)
  1. How to Upload Files to Server
  2. How to Upload Multiple Files to Server
  3. How to Upload a Dynamic Amount of Files to Server
  4. Upload Files with Progress
  5. Passing Multiple Parts Along a File with @PartMap
  6. How to Download Files from Server
  7. Download Files with Progress Updates
  8. How to Upload Files to Server
  1. Basic Authentication on Android
  2. Token Authentication on Android
  3. OAuth on Android
  4. Hawk Authentication on Android
  5. How to Refresh an Access Token
  1. Activate Response Caching (Etag, Last-Modified)
  2. Check Response Origin (Network, Cache, or Both)
  3. Force Server Cache Support with Response Interceptor
  4. Support App Offline Mode by Accessing Response Caches
  5. Analyze Cache Files
  1. Basics of Mocking Server Responses
  2. Customizing Network Behavior of Mocked Server Responses
  3. Mock APIs with JsonServer
  1. Callbacks
  2. Annotations
  3. Fluent Interface with Builders


Continue reading

University Enrollment Required

Future Students benefit from value packed videos and tutorials.

Enroll to receive exclusive content or sign in if you’re already a Future Student.

Enroll me for $15/mo

Norman Peitek

android & node.js dev

https://twitter.com/peitek

Explore the Library

Find interesting tutorials and solutions for your problems.

Android

Node.js

Server

Miscellaneous