nomemory / mockneat

MockNeat - the modern faker lib.

Mockneat is an arbitrary data-generator open-source library written in Java.

It provides a simple but powerful (fluent) API that enables developers to create json, xml, csv and sql data programatically.

It can also act as a powerful Random substitute or a mocking library.

Official Documentation: www.mockneat.com

Official Tutorial: www.mockneat.com

If you want to use mockneat to mock REST APIs checkout my other project: serverneat.

Installing

>= 0.4.4

Maven:

<dependency>
  <groupId>net.andreinc</groupId>
  <artifactId>mockneat</artifactId>
  <version>0.4.8</version>
</dependency>

Gradle:

implementation 'net.andreinc:mockneat:0.4.8'

<= 0.4.2

Maven:

<repositories>
    <repository>
        <id>jcenter</id>
        <url>https://jcenter.bintray.com/</url>
    </repository>
</repositories>
<dependencies>
    <dependency>
        <groupId>net.andreinc.mockneat</groupId>
        <artifactId>mockneat</artifactId>
        <version>0.4.2</version>
    </dependency>
</dependencies>

Gradle:

repositories {
  jcenter()
}
dependencies {
  compile 'net.andreinc.mockneat:mockneat:0.4.2'
}

Example - A random dice roll

List<String> somePeople = names().full().list(10).get();

fmt("#{person} rolled: #{roll1} #{roll2}")
            .param("person", seq(somePeople))
            .param("roll1", ints().rangeClosed(1, 6))
            .param("roll2", ints().rangeClosed(1, 6))
            .accumulate(10, "\n")
            .consume(System.out::println);

System.out.println("\nWho wins ?\n");

(possible) Output:

Sal Clouden rolled: 3 3
Cinthia Myrum rolled: 1 5
Wyatt Imber rolled: 5 1
Fidel Quist rolled: 2 2
Brandon Scrape rolled: 6 4
Arlene Cesare rolled: 6 4
Brandie Sumsion rolled: 3 4
Norris Tunby rolled: 3 5
Kareem Willoughby rolled: 1 5
Zoraida Finnerty rolled: 1 6

Who wins ?

Example - A simple CSV

System.out.println("First Name, Last Name, Email, Site, IP, Credit Card, Date");

csvs()
  .column(names().first())
  .column(names().last())
  .column(emails().domain("mockneat.com"))
  .column(urls().domains(POPULAR))
  .column(ipv4s().types(CLASS_B, CLASS_C_NONPRIVATE))
  .column(creditCards().types(AMERICAN_EXPRESS, VISA_16))
  .column(localDates().thisYear())
  .separator(" ; ")
  .accumulate(25, "\n")
  .consume(System.out::println);

(possible) Output:

Lorrie ; Urquilla ; [email protected] ; http://www.sugaredherlinda.com ; 172.150.99.65 ; 4991053014393849 ; 2019-05-25
Tabitha ; Copsey ; [email protected] ; http://www.arightcarnify.io ; 166.192.196.15 ; 4143903215740668 ; 2019-07-13
Laurine ; Patrylak ; [email protected] ; http://www.ninthbanc.gov ; 187.28.250.76 ; 4450754596171263 ; 2019-09-10
Starla ; Peiper ; [email protected] ; http://www.eathlessen.edu ; 202.189.115.252 ; 4470988734574428 ; 2019-02-18
Lakiesha ; Zevenbergen ; [email protected] ; http://www.unbendingeyes.edu ; 204.112.195.47 ; 4040555724781858 ; 2019-11-12

... and so on

---

Special thanks to the contributors:

• https://github.com/mariotrucco
• https://hatchful.shopify.com/ (for creating the logo)

mockneat's Issues

Industries is misspelled

mockneat/src/main/java/net/andreinc/mockneat/MockNeat.java

Line 410 in 60e53c0

public Industries inudstries() { return this.rIndustries; }

should be industries

Middle Name support to mockneat

Hi, I felt the need of this feature in one of my engagement and went ahead and developed that feature. I have also created a pull request to include it in main branch. This is the first time I am contributing a piece of code in an open source project. So, I am not sure if I did it correctly. Please let me know if I have made any mistakes/violated any rules while creating the PR

Incorrect documentation around secure random seeding

For particular use-cases, new MockNeat instances can be created by invoking the directly the public constructor. For example, if we want to create a secure MockNeat object with 123l as the seed:

Long seed = 123l;
// Note: RandomType.SECURE is the only type that supports seeding.
MockNeat mock = new MockNeat(RandomType.SECURE, seed);
It’s highly recommended to avoid creating multiple MockNeat instances. It’s better to stick with one instance per-project.

Two problems.

1. This isn't correct. // Note: RandomType.SECURE is the only type that supports seeding.

long hashBasis = 99823981L;
System.out.println(new MockNeat(RandomType.OLD, hashBasis).countries().names().get());
System.out.println(new MockNeat(RandomType.OLD, hashBasis).countries().names().get());
System.out.println(new MockNeat(RandomType.OLD, hashBasis).countries().names().get());
System.out.println(new MockNeat(RandomType.OLD, hashBasis).countries().names().get());

Prints out

Iceland
Iceland
Iceland
Iceland

2. You cannot successfully seed a SecureRandom instance. You can call setSeed() on it, but all that does is mix your seed in with the secure random's one. From my perspective I need the seed to be the same because I want random data, but I want random stable data in my tests. For this reason I would propose changing the docs to show that RandomType.OLD can be used if you want random mocked data that is stable.

ipv4s: add option to exclude private addresses

There is currently no way to make sure that ips returned by the ipv4s method won't be private, it would be nice to have an option for that

Requirement to create data for a retail industry...

Hi, I have a need to create a dataset for a retail industry where we need to create a list of edible and non-edible products. The use case is that based on the products bought by customers, they get rewards. This use case will also help to cross sell and up sell. I am thinking whether we should add a feature to create domain specific dataset. In this case it will produce a list of edible(like milk, meat) and non-edible(bath tissue etc).

Thanks

If you can use it easily in spring or spring boot, it would be fine.

Change int to long for number of records passed to list...

I will work on it and send a PR shortly. Currently the code takes only int for number of records. If we need billions of records created, we will need to change it to long

default MockUnit<List> list(Supplier<List> listSupplier, int size)

I am planning to add another method which will take long as an input.

Add MD, SHA MockUnitString

Adding a scale factor feature...

This is a request for one more feature. This feature will allow to add scale factor similar to what dsdgen has. I have done it in a crude way now. I am adding this feature so that we can track it for development if contributors and users think it will be a good feature.

Add unit tests for Probabilities.

Antivirus complains of Java Trojan for mockneat-0.3.8.jar

My antivirus complains of the following message for the mockneat library. The version I am using is 0.3.8.

Virus: Java.Trojan.GenericGBA.20288

Is there any possible changes that can be made in the next version of this library so as to avoid such for future versions?

localDates() does not work when using a formatter that contains time

This code:
localDates().thisYear().display(ISO_LOCAL_DATE_TIME)

fails, because ISO_LOCAL_DATE_TIME uses HOUR_OF_DAY, MINUTE_OF_HOUR, SECOND_OF_MINUTE but the formatted class is LocalDate which has no reference to time.

Content Type Dictionary

Great Library! Very easy to use

Would be great to have a Content-Type Mime dictionary to pull values from.

application/pdf

etc

Add the possibility to generate a random phone number

This is very common

bulk data vs sequential

Hi,

When we are trying to get data using set vs individually, they are different, even though seed value remain constant. The code snipped is attached here.

package net.andreinc.mockneat;

import net.andreinc.mockneat.types.enums.RandomType;

import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import java.util.stream.Stream;

public class TestApp {

  public static void main(String[] args) {
    MockNeat m =new MockNeat(RandomType.OLD, 1570704694342L);

     System.out.println(m.names().first().set(10).val());

    /* Stream<String> x = IntStream.range(0,10).mapToObj(i ->{
      return m.names().first().val();
    });

    List<String> result = x.collect(Collectors.toList());

    System.out.println(result);
    */
  }
}

Website search broken

Searching for anything in the website produces hits that look correct but the links presented do not work. They all look like this.

https://www.mockneat.com/undefined

Add unit tests for LongSeq

Seed feature to generate the same data again and again

I am reopening the seed feature request with below comments. This is in reply to Andrei question below

Do you recommend this seed feature so you can generate all the time the same data for tests / mock back-ends etc.?_

Rajib's response

Yes, I think this will be a required feature in certain scenarios. At least one I know where I need to test in different environments. If the data size is huge I would not like to copy the data to all environments rather I would run the datagen in all the environments to generate the same data.
I went through the code again. I noticed that the code is using Threadlocal which creates an internal seed and every run this seed will be different. I saw that secure random is there but even if I set it, it defaults to Threadlocal. What I did is, I commented all threadlocal and used securerandom. I then set the seed. Once I did that, I was able to generate the same data in every run. I know secure random will be slower than threadlocal but I think it will be good to have the flexibility to choose between the two.
Please let me know what you think about it. I can look at the code and make the necessary changes to make it flexible to choose between the two.

Add a new MockNeat method to return T objects with given probabilities.

Documentation on reflect default values.

Address with multiple lines

Add support for subclass reflection

Specifying subclass in mockNeat.reflect() method gives an error message which is "Please verify if the class has a public 'No Arguments' constructor".

I saw your code example at [https://stackoverflow.com/questions/26795661/how-to-generate-random-json-string-in-java] and I tried to generate data for my Patient class which contains field dob of type DOB. DOB is a subclass of Patient.

    public static void main(String[] args) throws IOException {
        MockNeat mockNeat = MockNeat.threadLocal();
        Gson gson = new GsonBuilder()
                .setPrettyPrinting()
                .create();

        String json = mockNeat
                .reflect(Patient.class)
                .field("firstName",mockNeat.names().first())
                .field("lastName", mockNeat.names().last())
                .field("dOB",mockNeat.reflect(Patient.DOB.class)
                        .field("dobDate",mockNeat.localDates().toUtilDate().toString())
                        )
                .field("status","Active")
                .map(gson::toJson)
                .val();

        System.out.println(json);
    }

Patient.java

public class Patient {

    public String lastName;
    public String firstName;
    public DOB dOB;

    public class DOB {
        public String dobDate;
        public Boolean estimated;
        public Boolean premature;
        public String duration;
        public String measure;

        public DOB() {
        }
    }
}

it gives me the following output

15:50:06.198 [main] INFO net.andreinc.mockneat.utils.file.FileManager - Loading internal dictionary 'last-names-american' in memory. The dictionary contains 88799 lines
15:50:06.240 [main] INFO net.andreinc.mockneat.utils.file.FileManager - Loading internal dictionary 'first-names-american-male' in memory. The dictionary contains 1219 lines
Exception in thread "main" java.lang.IllegalArgumentException: Cannot create an instance of 'com.pulse.framework.testDataTypes.Patient$DOB'. Please verify if the class has a public 'No Arguments' constructor: com.pulse.framework.testDataTypes.Patient$DOB.
	at net.andreinc.mockneat.unit.objects.Reflect.instance(Reflect.java:145)
	at net.andreinc.mockneat.unit.objects.Reflect.lambda$supplier$0(Reflect.java:86)
	at net.andreinc.mockneat.abstraction.MockUnitValue.get(MockUnitValue.java:43)
	at net.andreinc.mockneat.unit.objects.Reflect.lambda$setValues$2(Reflect.java:162)
	at java.util.ArrayList.forEach(ArrayList.java:1249)
	at net.andreinc.mockneat.unit.objects.Reflect.setValues(Reflect.java:150)
	at net.andreinc.mockneat.unit.objects.Reflect.lambda$supplier$0(Reflect.java:87)
	at net.andreinc.mockneat.abstraction.MockUnit.lambda$map$0(MockUnit.java:161)
	at net.andreinc.mockneat.abstraction.MockUnit.val(MockUnit.java:64)
	at com.pulse.pulseCloud.presetup.datafaker.MockNeatSample.main(MockNeatSample.java:31)
Caused by: java.lang.NoSuchMethodException: com.pulse.framework.testDataTypes.Patient$DOB.<init>()
	at java.lang.Class.getConstructor0(Class.java:3082)
	at java.lang.Class.getDeclaredConstructor(Class.java:2178)
	at net.andreinc.mockneat.unit.objects.Reflect.instance(Reflect.java:140)
	... 9 more

Ability to create a lambda(or something) of methods that will be have N of those methods called

Any thoughts on the ability to have some sort of listing of methods or a lambda/builder that allows you to define a probability that the method will be called, and be able to set a Min and/or max methods to be called.

Such as: I have 4 snippets of code: I want each snippet have a probability of 50%, and I want that a minimum of 1 method will always be called, but which will be random).

Generate Numbers with a lower bound?

it seems to be possible to generate a Upper bound using the .bound() method when working with Numbers. But is it possible to set a lower bound? The use case is mainly to enforce positive numbers

1. >= 0
2. > 0

using list(MockUnitInt) doesn't generate a new random value each time.

I am trying to generate lists of data with random size. I started with

MockUnit<T> things;
MockUnit<List<T>> listsOfThings = things.list(ints().range(0, 10));

my expectation from looking at the API was that each time I called listOfThings.val() I'd get a new list with a random size. in practice, the random size is evaluated once, and I get lists of the same size each time.

i've worked around this by doing

MockUnit<List<T>> listsOfThings = ints().range(0, 10).map( size -> things.list(size).val());

but that feels awkward.

what's the intention of MockUnit.list(MockUnitInt) ? would you be open to a PR that evaluates the random size each time a new val is requested?

Markovs doesn't honor RandomType

I have a use case to have repeatable text generation based on a given seed. It appears that Markovs doesn't honor RandomType.OLD as it directly relies on ThreadLocalRandom. Please let me know if you would like me to create a pull request to fix it.

Add the possibility to register Custom MockUnits to the MockNeat object.

Write a generic mechanism that allows the user to register Custom MockUnits with a MockNeat instance and/or globally.

mock.register("name", Class, MockUnit)
mock.custom("name").val

To investigate what is the best strategy to achieve this.

Add Maven Central / JCenter Integration

Web API data sources

There exist services like https://randomuser.me/ which provide restful api to get random data in json format, thus not requiring a local database. Would it be too far from this project idea to incorporate a generic rest api in order to get data from the web? Something like:

MockNeat.threadLocal()
         .fromRest("/someurl")
         .params(...)
         .deserializer(SomeDeserializer.class)
         .val()

subsequent run data consistency

Thanks a lot for making such an excellent Mockneat library. We have a requirement where every time (subsequent run) the application should generate the same data from the MockNeat. Please let me know if MockNeat supports such features.

I tested the following code

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import net.andreinc.mockneat.MockNeat;
import net.andreinc.mockneat.types.enums.RandomType;

import java.util.List;

public class TestApp {

  public static void main(String[] args) {

    MockNeat m = new MockNeat(RandomType.SECURE,8L);

    List<Date1> date = m.reflect(Date1.class)
            .field("dateId",m.intSeq().start(1).increment(1))
            .field("year",m.ints().range(2000, 2019))
            .field("month",m.months().mapToString())
            .field("day",m.ints().range(1, 28))
            .list((int) 10)
            .val();

    Gson gson = new GsonBuilder().create();
    System.out.println(gson.toJson(date));

  }
}

public class Date1 {

  private Integer dateId;
  private Integer year;
  private String month;
  private Integer day;

  public Integer getDateId() {
    return dateId;
  }
  public void setDateId(Integer dateId) {
    this.dateId = dateId;
  }
  public Integer getYear() {
    return year;
  }
  public void setYear(Integer year) {
    this.year = year;
  }
  public String getMonth() {
    return month;
  }
  public void setMonth(String month) {
    this.month = month;
  }
  public Integer getDay() {
    return day;
  }
  public void setDay(Integer day) {
    this.day = day;
  }

  public Date1() {

  }
}

When I run this code , in each subsequent run I always find one record is not matching with old set of record.

Code coverage

I see you have extensive tests written but its hard to try and figure out how well the tests cover all of the code. Is it possible to get some code coverage stats on the readme?

Im potentially looking to use this in my tests and so far i havent really found another library that is fit for purpose and easy to use such as yours.

Add support to convert JSON Schema draft to sample JSON data

Add support to convert JSON Schema lets say in draft 6 to generate random data.
User will provide JSON Schema in draft and the library should return random data.
This would help in mocking test data from swagger directly.

Composite Object MockNeat

I am trying to create mock data from MockNeat for Composite Object for example (It is scala code but same applies for the Java Beans as well):

case class User(name:String)

case class Group(id:Int, name:String)

case class UserGroups(user:User, group:Group)

I would like to create data for UserGroups. I tried the following approach:

1. generate User mock data using User class --> userData
2. generate Group data using using Group Class --> groupData
3. Generate UserGroup using mockNeat.from(userData) , mockNeat.from(groupData)

The point#3 is not working, It keep throwing error HashSet empty.

Add a MockUnitString for generating SSCs (US Social Security Codes)

Issue with mocking a class with a final static long field

Hello,
I have a class which implements the Serializable interface and has the default declaration "serialVersionUID"
When I tried to mock the class, I get the error 'Cannot set field long.serialVersionUID to null

Stack trace below:
java.lang.IllegalArgumentException: Cannot set field long.serialVersionUID with value 'null'. Is the supplied value correct ? at net.andreinc.mockneat.unit.objects.Reflect.lambda$setValues$2(Reflect.java:176) at java.util.ArrayList.forEach(ArrayList.java:1257) at net.andreinc.mockneat.unit.objects.Reflect.setValues(Reflect.java:150) at net.andreinc.mockneat.unit.objects.Reflect.lambda$supplier$0(Reflect.java:87) at net.andreinc.mockneat.abstraction.MockUnit.val(MockUnit.java:68)

Add Unit tests for default fields in the Reflect MockUnit

Unable to use PostgreSQL escaper with SQLInserts

The method public SQLInserts column(String column, MockUnit mockUnit, UnaryOperator<String> sqlFormatter) in class net.andreinc.mockneat.unit.text.SQLInserts does not work with the Postgres and MySQL escapers, since both escaper return a Function and not a UnaryOperator.

This is in contrast with the documentation here: https://www.mockneat.com/tutorial#sql-inserts

As a temporary fix, I have created a local copy of PostgreSQL that uses UnaryOperator.

Feature to be able to generate data in chunks with a seed...

This is not a issue but a feature addition. I will try to see if I can add it but please feel free if anyone wants to pick it up. The feature requirement is as below

Lets say that I want to create a creditcard file with 10,000 records. I should be able to create the file in one run or in 5 runs with 2000 records in each run. When I compare the one file with 10000 records and 5 files with 2000 records each, the content should be the same.

I am not sure if this feature is already available in mockneat

Allow option to provide BIN prefix when generating credit card number

Congratulations on a cool library, "neat" is an appropriate name. 👍

I was planning to use the creditCards() method to easily generate a credit card number, but need it to be generated with a specific BIN else it will be rejected as invalid in the system I'm testing. From the code the card prefix is taken from the CreditCardType enum, which in the case of Visa is just the first digit (4). From what I've read this is the IIN, the issuer identification number.

It would be nice if CreditCards had another method to specify a pool of BINs to use when generating the number, e.g.

String cardNumber1 = mockNeat.creditCards().visa().bins(445434).val();
String cardNumber2 = mockNeat.creditCards().visa().bins(445434, 414433).val();

Cannot instantiate Mockneat

In my env I've got JDK-11, Gradle 4.10.2 and Mockneat 0.2.4.
Due to bug in Gradle gradle/gradle#2657 Mockneat not allow to create instance.
Is it possible to have logging system optional or make it pluggable with standard JDK-11 eco system?

provide more kotlin (and scala?) friendly supplier.val()

val email = mock.emails().`val`()

^-- its not bad, but maybe an alias?

Support for random length strings

When we call:

mock.fmt("SOME_PREFIX_#{randomsuffix}")
.param("randomsuffix", mock.strings().size(
    mock.ints().range(42, 62).val()
))

the library naturally generates a random int between 42 and 62 once and then generates randomsuffix with this length.

But the problem is, it doesn't vary between calls because mock.strings().size() method only accepts an int parameter (and not a MockUnit) so we can't pass the method call without the .val() call made.

Can you please add support for variable length strings? Perhaps for every other API methods this applies to?

Fix MoneyTest.testMoneyBound

Investigate why the test is failing from time to time and if there are any other implications.

Issue importing lib in a gradle build

Gradle build fails with error:
org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find net.andreinc.mockneat:mockneat:0.1.7.

Similar error for previous versions too.

Ability to generate Times?

Seems you can generate Dates, but is is possible to generic Dates with Times ?

Add a MockUnitString for generating IBAN codes

mockneat unable to mock final static members of target mock object

Java beans in my projects are serializable and all have serialVersonUID which are final. When I try to mock that object, It gives me IllegalArgumentException: Field is marked as final. It can't be modified. Please remove it from field list. I checked if there is any option to ignore final members while creating mockneat object. But I couldn't find one.

Can't import the library from jcenter Maven repo

I tried to import this library from jcenter Maven repo using these configs in pom.xml but it didn't work:

	<repositories>
	 <repository>
      <id>jcenter</id>
      <url>http://jcenter.bintray.com </url>
      <snapshots>
        <enabled>true</enabled>
        <updatePolicy>never</updatePolicy>
        <checksumPolicy>warn</checksumPolicy>
      </snapshots>
       <releases>
         <enabled>true</enabled>
         <checksumPolicy>warn</checksumPolicy>
      </releases>
   </repository>
	</repositories>
 
   <dependency>
  <groupId>net.andreinc.mockneat</groupId>
  <artifactId>mockneat</artifactId>
  <version>0.1.0</version>
  <type>pom</type>
</dependency>

Is there something wrong with the config?

No instructions to use this project from Maven?

It looks too complex to use this project from Maven, since it's required to download two different source projects, build them and add them to local Maven repository.

Almost useless when working with Maven.

Add possibility to directly mock data into database.

Just looking at the tutorial for mocking a MySQL database i had the thought that it would make things even better if you could directly store the resulting list into a database. A JPA provider could take care of generalising the sql code generation based on the the domain classes and the mockneat list().val(); call can make a call to the JPA persist method for storing.

Im not familiar with the functional programming api in java 8 but if you can tell me what would be needed in order to integrate something like this into Mockneat i can help with that part.