Skip to content
GitLab
Unverified Commit 9f969666 authored by Greg Messner's avatar Greg Messner Committed by GitHub
Browse files

Java 8 Stream Documentation improvements (#298) 

* Added test for lazy Stream evaluation (#295).
* Cleaned up Stream support documentaion (#295).
parent ce32db4c
Hide whitespace changes
Inline Side-by-side
README.md
View file @ 9f969666
......@@ -11,7 +11,7 @@ To utilize the GitLab API for Java in your project, simply add the following dep
```java
dependencies {
...
compile group: 'org.gitlab4j', name: 'gitlab4j-api', version: '4.9.15'
compile group: 'org.gitlab4j', name: 'gitlab4j-api', version: '4.9.16'
}
```
......@@ -22,7 +22,7 @@ dependencies {
<dependency>
<groupId>org.gitlab4j</groupId>
<artifactId>gitlab4j-api</artifactId>
<version>4.9.15</version>
<version>4.9.16</version>
</dependency>
```
......@@ -141,34 +141,41 @@ List<Project> allProjects = projectPager.all();
---
## Java 8 Stream Support
As of GitLab4J-API 4.9.2, you can also stream list based items in a Java 8 Stream using a Pager instance as follows:
```java
// Get a Pager instance to get a Stream<Project> instance.
Pager<Project> projectPager = gitlabApi.getProjectsApi().getProjects(10);
As of GitLab4J-API 4.9.2, all GitLabJ-API methods that return a List result also similarlly named method returns a Java 8 Stream. The Stream returning methods use the following naming convention: ```getXxxxxStream()```.
**IMPORTANT**
The built-in methods that return a Stream do so using ___eager evaluation___, meaning all items are pre-fetched from the GitLab server and a Stream is returned which will stream those items. **Eager evaluation does NOT support paralell reading of data from ther server, it does however allow for paralell processing of the Stream post data fetch.**
To stream using ___lazy evaluation___, use the GitLab4J-API methods that return a ```Pager``` instance, and then call the ```lazyStream()``` method on the ```Pager``` instance to create a lazy evaluation Stream. The Stream utilizes the ```Pager``` instance to page through the available items. **A lazy Stream does NOT support parallel operations or skipping.**
**Eager evaluation example usage:**
// Stream the Projects printing out the project name.
projectPager.stream().map(Project::getName).forEach(name -> System.out.println(name));
```
The following API classes also include ```getXxxxxStream()``` methods which return a Java 8 Stream:
```
GroupApi
IssuesApi
NotesApi
ProjectApi
RepositoryApi
TagsApi
UserApi
```
Example usage:
```java
// Stream the visible Projects printing out the project name.
gitlabApi.getProjectsApi().getProjectsStream().map(Project::getName).forEach(name -> System.out.println(name));
// Stream the visible projects printing out the project name.
Stream<Project> projectStream = gitlabApi.getProjectApi().getProjectsStream();
projectStream.map(Project::getName).forEach(name -> System.out.println(name));
// Operate on the stream in parallel, this example sorts User instances by username
// NOTE: Fetching of the users is not done in paralell,
// only the soprting of the users is a paralell operation.
Stream<User> stream = new UserApi(gitLabApi).getUsersStream();
List<User> sortedUsers = stream.parallel().sorted(comparing(User::getUsername)).collect(toList());
List<User> users = stream.parallel().sorted(comparing(User::getUsername)).collect(toList());
```
**Lazy evaluation example usage:**
```java
// Get a Pager instance to that will be used to lazily stream Project instances.
// In this example, 10 Projects per page will be pre-fetched.
Pager<Project> projectPager = gitlabApi.getProjectApi().getProjects(10);
// Lazily stream the Projects, printing out each project name, limit the output to 5 project names
projectPager.lazyStream().limit(5).map(Project::getName).forEach(name -> System.out.println(name));
```
---
## Java 8 Optional&lt;T&gt; Support
GitLab4J-API supports Java 8 Optional&lt;T&gt; for API calls that result in the return of a single item. Here is an example on how to use the Java 8 Optional&lt;T&gt; API calls:
......
src/main/java/org/gitlab4j/api/GitLabApi.java
View file @ 9f969666
......@@ -27,7 +27,7 @@ public class GitLabApi {
private final static Logger LOGGER = Logger.getLogger(GitLabApi.class.getName());
/** GitLab4J default per page. GitLab will ignore anything over 100. */
public static final int DEFAULT_PER_PAGE = 100;
public static final int DEFAULT_PER_PAGE = 96;
/** Specifies the version of the GitLab API to communicate with. */
public enum ApiVersion {
......@@ -1393,7 +1393,7 @@ public class GitLabApi {
return (optional.get());
}
/**
* Gets the SnippetsApi instance owned by this GitLabApi instance. The SnippetsApi is used
* to perform all snippet related API calls.
......
src/main/java/org/gitlab4j/api/Pager.java
View file @ 9f969666
......@@ -18,12 +18,12 @@ import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
/**
* <p>This class defines an Iterator implementation that is used as a paging iterator for all API methods that
* <p>This class defines an Iterator implementation that is used as a paging iterator for all API methods that
* return a List of objects. It hides the details of interacting with the GitLab API when paging is involved
* simplifying accessing large lists of objects.</p>
*
*
* <p>Example usage:</p>
*
*
* <pre>
* // Get a Pager instance that will page through the projects with 10 projects per page
* Pager&lt;Project&gt; projectPager = gitlabApi.getProjectsApi().getProjectsPager(10);
......@@ -35,8 +35,8 @@ import com.fasterxml.jackson.databind.ObjectMapper;
* System.out.println(project.getName() + " : " + project.getDescription());
* }
* }
* </pre>
*
* </pre>
*
* @param <T> the GitLab4J type contained in the List.
*/
public class Pager<T> implements Iterator<List<T>>, Constants {
......@@ -48,6 +48,7 @@ public class Pager<T> implements Iterator<List<T>>, Constants {
private List<String> pageParam = new ArrayList<>(1);
private List<T> currentItems;
private Stream<T> pagerStream = null;
private AbstractApi api;
private MultivaluedMap<String, String> queryParams;
......@@ -59,7 +60,7 @@ public class Pager<T> implements Iterator<List<T>>, Constants {
/**
* Creates a Pager instance to access the API through the specified path and query parameters.
*
*
* @param api the AbstractApi implementation to communicate through
* @param type the GitLab4J type that will be contained in the List
* @param itemsPerPage items per page
......@@ -114,7 +115,7 @@ public class Pager<T> implements Iterator<List<T>>, Constants {
/**
* Get the specified integer header value from the Response instance.
*
*
* @param response the Response instance to get the value from
* @param key the HTTP header key to get the value for
* @return the specified integer header value from the Response instance
......@@ -136,7 +137,7 @@ public class Pager<T> implements Iterator<List<T>>, Constants {
/**
* Sets the "page" query parameter.
*
*
* @param page the value for the "page" query parameter
*/
private void setPageParam(int page) {
......@@ -204,7 +205,7 @@ public class Pager<T> implements Iterator<List<T>>, Constants {
/**
* This method is not implemented and will throw an UnsupportedOperationException if called.
*
*
* @throws UnsupportedOperationException when invoked
*/
@Override
......@@ -289,7 +290,7 @@ public class Pager<T> implements Iterator<List<T>>, Constants {
throw new RuntimeException(e);
}
}
/**
* Gets all the items from each page as a single List instance.
*
......@@ -312,34 +313,62 @@ public class Pager<T> implements Iterator<List<T>>, Constants {
}
/**
* Builds and returns a Stream instance for streaming all the items from each page.
* Builds and returns a Stream instance which is pre-populated with all items from all pages.
*
* @return a Stream instance for streaming all the items from each pag
* @throws GitLabApiException if any error occurs
* @return a Stream instance which is pre-populated with all items from all pages
* @throws IllegalStateException if Stream has already been issued
* @throws GitLabApiException if any other error occurs
*/
public Stream<T> stream() throws GitLabApiException {
public Stream<T> stream() throws GitLabApiException, IllegalStateException {
// Make sure that current page is 0, this will ensure the whole list is streamed
// regardless of what page the instance is currently on.
currentPage = 0;
if (pagerStream == null) {
synchronized (this) {
if (pagerStream == null) {
// Create a Stream.Builder to contain all the items. This is more efficient than
// getting a List with all() and streaming that List
Stream.Builder<T> streamBuilder = Stream.builder();
// Make sure that current page is 0, this will ensure the whole list is streamed
// regardless of what page the instance is currently on.
currentPage = 0;
// Iterate through the pages and append each page of items to the stream builder
while (hasNext()) {
next().forEach(streamBuilder);
}
// Create a Stream.Builder to contain all the items. This is more efficient than
// getting a List with all() and streaming that List
Stream.Builder<T> streamBuilder = Stream.builder();
return (streamBuilder.build());
// Iterate through the pages and append each page of items to the stream builder
while (hasNext()) {
next().forEach(streamBuilder);
}
pagerStream = streamBuilder.build();
return (pagerStream);
}
}
}
throw new IllegalStateException("Stream already issued");
}
public Stream<T> lazyStream() {
// Make sure that current page is 0, this will ensure the whole list is streamed
// regardless of what page the instance is currently on.
currentPage = 0;
/**
* Creates a Stream instance for lazily streaming items from the GitLab server.
*
* @return a Stream instance for lazily streaming items from the GitLab server
* @throws IllegalStateException if Stream has already been issued
*/
public Stream<T> lazyStream() throws IllegalStateException {
if (pagerStream == null) {
synchronized (this) {
if (pagerStream == null) {
// Make sure that current page is 0, this will ensure the whole list is streamed
// regardless of what page the instance is currently on.
currentPage = 0;
pagerStream = StreamSupport.stream(new PagerSpliterator<T>(this), false);
return (pagerStream);
}
}
}
return StreamSupport.stream(new PagerSpliterator<T>(this), false);
throw new IllegalStateException("Stream already issued");
}
}
src/main/java/org/gitlab4j/api/ProjectApi.java
View file @ 9f969666
......@@ -818,7 +818,7 @@ public class ProjectApi extends AbstractApi implements Constants {
if (isApiVersion(ApiVersion.V3)) {
boolean isPublic = (project.getPublic() != null ? project.getPublic() : project.getVisibility() == Visibility.PUBLIC);
formData.withParam("public", isPublic);
if (project.getTagList() != null && !project.getTagList().isEmpty()) {
throw new IllegalArgumentException("GitLab API v3 does not support tag lists when creating projects");
}
......@@ -826,7 +826,7 @@ public class ProjectApi extends AbstractApi implements Constants {
Visibility visibility = (project.getVisibility() != null ? project.getVisibility() :
project.getPublic() == Boolean.TRUE ? Visibility.PUBLIC : null);
formData.withParam("visibility", visibility);
if (project.getTagList() != null && !project.getTagList().isEmpty()) {
formData.withParam("tag_list", String.join(",", project.getTagList()));
}
......@@ -1057,7 +1057,7 @@ public class ProjectApi extends AbstractApi implements Constants {
formData.withParam("visibility_level", project.getVisibilityLevel());
boolean isPublic = (project.getPublic() != null ? project.getPublic() : project.getVisibility() == Visibility.PUBLIC);
formData.withParam("public", isPublic);
if (project.getTagList() != null && !project.getTagList().isEmpty()) {
throw new IllegalArgumentException("GitLab API v3 does not support tag lists when updating projects");
}
......@@ -1065,7 +1065,7 @@ public class ProjectApi extends AbstractApi implements Constants {
Visibility visibility = (project.getVisibility() != null ? project.getVisibility() :
project.getPublic() == Boolean.TRUE ? Visibility.PUBLIC : null);
formData.withParam("visibility", visibility);
if (project.getTagList() != null && !project.getTagList().isEmpty()) {
formData.withParam("tag_list", String.join(",", project.getTagList()));
}
......@@ -1090,7 +1090,7 @@ public class ProjectApi extends AbstractApi implements Constants {
/**
* Forks a project into the user namespace of the authenticated user or the one provided.
* The forking operation for a project is asynchronous and is completed in a background job.
* The forking operation for a project is asynchronous and is completed in a background job.
* The request will return immediately.
*
* <pre><code>POST /projects/:id/fork</code></pre>
......@@ -1109,7 +1109,7 @@ public class ProjectApi extends AbstractApi implements Constants {
/**
* Forks a project into the user namespace of the authenticated user or the one provided.
* The forking operation for a project is asynchronous and is completed in a background job.
* The forking operation for a project is asynchronous and is completed in a background job.
* The request will return immediately.
*
* <pre><code>POST /projects/:id/fork</code></pre>
......@@ -1130,7 +1130,7 @@ public class ProjectApi extends AbstractApi implements Constants {
* Create a forked from/to relation between existing projects.
*
* <pre><code>POST /projects/:id/fork/:forkFromId</code></pre>
*
*
*
* @param projectIdOrPath projectIdOrPath the project in the form of an Integer(ID), String(path), or Project instance
* @param forkedFromId the ID of the project that was forked from
......@@ -1455,7 +1455,7 @@ public class ProjectApi extends AbstractApi implements Constants {
}
/**
* Get a Pager of project users matching the specified search string. This Pager includes
* Get a Pager of project users matching the specified search string. This Pager includes
* all project members and all users assigned to project parent groups.
*
* <pre><code>GET /projects/:id/users</code></pre>
......@@ -1644,7 +1644,7 @@ public class ProjectApi extends AbstractApi implements Constants {
* @return the added ProjectHook instance
* @throws GitLabApiException if any exception occurs
*/
public ProjectHook addHook(String projectName, String url, ProjectHook enabledHooks, boolean enableSslVerification, String secretToken)
public ProjectHook addHook(String projectName, String url, ProjectHook enabledHooks, boolean enableSslVerification, String secretToken)
throws GitLabApiException {
if (projectName == null) {
......@@ -2266,9 +2266,9 @@ public class ProjectApi extends AbstractApi implements Constants {
/**
* Get a list of projects that were forked from the specified project.
*
*
* <pre><code>GET /projects/:id/forks</code></pre>
*
*
* @param projectIdOrPath projectIdOrPath the project in the form of an Integer(ID), String(path), or Project instance, required
* @return a List of forked projects
* @throws GitLabApiException if any exception occurs
......@@ -2382,7 +2382,7 @@ public class ProjectApi extends AbstractApi implements Constants {
}
/**
* Uploads and sets the project avatar for the specified project.
* Uploads and sets the project avatar for the specified project.
*
* <pre><code>PUT /projects/:id/uploads</code></pre>
*
......@@ -2395,4 +2395,4 @@ public class ProjectApi extends AbstractApi implements Constants {
Response response = putUpload(Response.Status.OK, "avatar", avatarFile, "projects", getProjectIdOrPath(projectIdOrPath));
return (response.readEntity(Project.class));
}
}
\ No newline at end of file
}
src/test/java/org/gitlab4j/api/TestStreams.java
View file @ 9f969666
......@@ -5,6 +5,7 @@ import static org.gitlab4j.api.JsonUtils.compareJson;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
import static org.junit.Assume.assumeTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import static org.mockito.MockitoAnnotations.initMocks;
......@@ -85,4 +86,36 @@ public class TestStreams implements Constants {
assertTrue(compareJson(sortedUsers.get(i), users.get(i)));
}
}
@Test
public void testLazyStream() throws Exception {
// Arrange
Pager<User> pager = new UserApi(gitLabApi).getUsers(10);
Stream<User> stream = pager.lazyStream();
// Assert
assertNotNull(stream);
List<String> usernames = stream.map(User::getUsername).collect(toList());
assertNotNull(usernames);
assertEquals(usernames.size(), sortedUsers.size());
for (int i = 0; i < sortedUsers.size(); i++) {
assertTrue(usernames.contains(sortedUsers.get(i).getUsername()));
}
}
@Test
public void testStreamLazyLimit() throws Exception {
// Arrange and only continue if there are more than 3 users
Pager<User> pager = new UserApi(gitLabApi).getUsers(2);
assumeTrue(pager != null && pager.getTotalItems() > 3);
Stream<User> stream = pager.lazyStream();
// Assert
List<User> users = stream.limit(3).collect(toList());
assertNotNull(users);
assertEquals(3, users.size());
}
}
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment