org.openimaj.io

Class HttpUtils

java.lang.Object

org.openimaj.io.HttpUtils

public class HttpUtils
extends Object

HTTP(S) download utilities, with support for HTTP redirects and meta refresh redirection.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HttpUtils.MetaRefreshRedirectStrategy A RedirectStrategy that can deal with meta-refresh style redirection

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_USERAGENT The default user-agent string

Method Summary

Modifier and Type	Method and Description
static InputStream	readURL(URL url) Open an HttpURLConnection to the URL as an array of bytes.
static InputStream	readURL(URL url, boolean followRedirects) Open an HttpURLConnection to the URL as an array of bytes.
static <T extends org.openimaj.io.InternalReadable> T	readURL(URL url, Class<? extends T> clz) Read the an object from the given URL.
static <T,Q extends InputStreamObjectReader<T>> T	readURL(URL url, Q reader) Read the an object from the given URL.
static <T extends org.openimaj.io.InternalReadable> T	readURL(URL url, T obj) Read the internal state of an object from the given URL.
static IndependentPair<org.apache.http.HttpEntity,ByteArrayInputStream>	readURLAsByteArrayInputStream(URL u, boolean followRedirects) Read the contents of the given URL as a ByteArrayInputStream (i.e.
static IndependentPair<org.apache.http.HttpEntity,ByteArrayInputStream>	readURLAsByteArrayInputStream(URL url, int connectionTimeout, int readTimeout, org.apache.http.client.RedirectStrategy redirectStrategy, String userAgent) Read the contents of the given URL as a ByteArrayInputStream (i.e.
static IndependentPair<org.apache.http.HttpEntity,ByteArrayInputStream>	readURLAsByteArrayInputStream(URL u, org.apache.http.client.RedirectStrategy strategy) Read the contents of the given URL as a ByteArrayInputStream (i.e.
static byte[]	readURLAsBytes(URL u) Read the contents of the given URL as an array of bytes.
static byte[]	readURLAsBytes(URL u, boolean followRedirects) Read the contents of the given URL as an array of bytes.
static InputStream	readURLAsStream(URL url) Open a InputStream to the contents referenced by the URL.
static InputStream	readURLAsStream(URL url, boolean followRedirects) Open a InputStream to the contents referenced by the URL.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_USERAGENT

public static final String DEFAULT_USERAGENT

The default user-agent string

See Also:

Constant Field Values

Method Detail

readURLAsBytes

public static byte[] readURLAsBytes(URL u)
                             throws IOException

Read the contents of the given URL as an array of bytes. Redirects are followed automatically.

Parameters:

u - the URL to read from

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURLAsBytes

public static byte[] readURLAsBytes(URL u,
                                    boolean followRedirects)
                             throws IOException

Read the contents of the given URL as an array of bytes. If redirects are not being followed, then the result will be null if the URL is redirected.

Parameters:

u - the URL to read from

followRedirects - should redirects be followed?

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURLAsByteArrayInputStream

public static IndependentPair<org.apache.http.HttpEntity,ByteArrayInputStream> readURLAsByteArrayInputStream(URL u,
                                                                                                             boolean followRedirects)
                                                                                                      throws IOException

Read the contents of the given URL as a ByteArrayInputStream (i.e. a byte[] in memory wrapped in an InputStream). If redirects are not being followed, then the result will be null if the URL is redirected.

Parameters:

u - the URL to read from

followRedirects - should redirects be followed?

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURLAsByteArrayInputStream

public static IndependentPair<org.apache.http.HttpEntity,ByteArrayInputStream> readURLAsByteArrayInputStream(URL u,
                                                                                                             org.apache.http.client.RedirectStrategy strategy)
                                                                                                      throws IOException

Read the contents of the given URL as a ByteArrayInputStream (i.e. a byte[] in memory wrapped in an InputStream). If redirects are not being followed, then the result will be null if the URL is redirected.

Parameters:

u - the URL to read from

strategy - how redirects should be followed

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURLAsByteArrayInputStream

public static IndependentPair<org.apache.http.HttpEntity,ByteArrayInputStream> readURLAsByteArrayInputStream(URL url,
                                                                                                             int connectionTimeout,
                                                                                                             int readTimeout,
                                                                                                             org.apache.http.client.RedirectStrategy redirectStrategy,
                                                                                                             String userAgent)
                                                                                                      throws IOException

Read the contents of the given URL as a ByteArrayInputStream (i.e. a byte[] in memory wrapped in an InputStream). If redirects are not being followed, then the result will be null if the URL is redirected.

Parameters:

url - the URL to read from

connectionTimeout - amount of time to wait for connection

readTimeout - amount of time to wait for reading

redirectStrategy - the redirection strategy

userAgent - the useragent string

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURL

public static InputStream readURL(URL url)
                           throws IOException

Open an HttpURLConnection to the URL as an array of bytes. Redirects are followed automatically.

Parameters:

url - the URL to read from

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURL

public static InputStream readURL(URL url,
                                  boolean followRedirects)
                           throws IOException

Open an HttpURLConnection to the URL as an array of bytes.

Parameters:

url - the URL to read from

followRedirects - should redirects be followed?

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURLAsStream

public static InputStream readURLAsStream(URL url)
                                   throws IOException

Open a InputStream to the contents referenced by the URL. Redirects are followed automatically.

Parameters:

url - the URL to read from

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURLAsStream

public static InputStream readURLAsStream(URL url,
                                          boolean followRedirects)
                                   throws IOException

Open a InputStream to the contents referenced by the URL. If redirects are not being followed, then the result will be null if the URL is redirected.

Parameters:

url - the URL to read from

followRedirects - should redirects be followed.

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURL

public static <T extends org.openimaj.io.InternalReadable> T readURL(URL url,
                                                                     T obj)
                                                              throws IOException

Read the internal state of an object from the given URL.

Type Parameters:

T - Type of object being read.

Parameters:

url - the URL to read from

obj - the object to fill

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURL

public static <T extends org.openimaj.io.InternalReadable> T readURL(URL url,
                                                                     Class<? extends T> clz)
                                                              throws IOException

Read the an object from the given URL.

Type Parameters:

T - Type of object being read.

Parameters:

url - the URL to read from

clz - the class of the object to read

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL

readURL

public static <T,Q extends InputStreamObjectReader<T>> T readURL(URL url,
                                                                 Q reader)
                                                          throws IOException

Read the an object from the given URL.

Type Parameters:

T - Type of object being read.

Q - Type of the object reader.

Parameters:

url - the URL to read from

reader - the reader that creates the object.

Returns:

the content referenced by the URL

Throws:

IOException - if an error occurs

IllegalArgumentException - if the URL is not an HTTP(s) URL