How to create dummy data with different datatypes in python? [closed]

Question

I am trying to create a logistics dummy dataset for doing some analysis and possible predictions on the data.

Assumed variables are as follows:

 VARIABLES                     RANGES
 awb                           random number eg:235533
 destination_city              random cities
 product                       different products
 product_category              different categories
 origin_city                   random metro cities
 logistics_provider_id         id's eg:1,20,28,27
 dispatch_date                 datetime between mar01-2015 to mar15-2015
 final_delivery_status         created,delivered,returned
 actual_delivery_date          datetime between mar16-2015 to mar30-2015
 promised_delivery_date        datetime between mar25-2015 to Apr6-2015

So, from the above variables assumed i want to create the dummy data with in the ranges mentioned. How can i create the dummy data using python

Expected output:

example_dummy_data:

  awb        destination_city   product               product_category
1 104842891  Byatarayanapura    Wrangler Denim Jeans  Men's Clothing
2 104842938  Bareilly           Sky Blue Denim        Men's Clothing
3 104842942  Saharanpur         puma shoes            Men's Footwear
4 104842943  Saharanpur         classic puma shoes    Men's Foorwear
5 104843066  Mumbai             Elegant black belt    Fashion Accessories

  origin_city  log_prov_id   dispatch date          final_del_status
1 Gurgaon      18            2014-09-02 00:26:11       DEL
2 Bangalore    19            2014-09-01 23:34:30       RTN
3 New Delhi    18            2014-09-01 18:59:41       RTC
4 New Delhi    15            2014-09-02 00:05:33       DEL
5 Hyderabad    16            2014-09-01 22:09:14       UDL

  Actual_del_date        promised_del_date
1 2014-09-03 00:00:00   2014-09-05 20:00:00
2 2014-09-04 00:00:00   2014-09-06 20:00:00
3 2014-09-04 00:00:00   2014-09-06 20:00:00
4 2014-09-04 00:00:00   2014-09-07 20:00:00
5 2014-09-02 00:00:00   2014-09-06 20:00:00

I want to create the data with 10000 rows like above,Is there any best way to create in the ranges mentioned above

Tried:

import random
a = [int(10000*random.random()) for i in xrange(10000)]

Found how to generate random numbers but not with in the ranges and cities i want. So Please help me how to create the dummy data like i have mentioned with 10000 rows with in the ranges i have mentioned.

Answer 1

The faker package is built for this kind of use-case. It handles the names, integers and dates already, although you will probably want to add your own products and product categories.

import pandas
from faker import Factory
import random

faker = Factory.create()
status = 'created,delivered,returned'.split(',')

def date_between(d1, d2):
    f = '%b%d-%Y'
    return faker.date_time_between_dates(datetime.strptime(d1, f), datetime.strptime(d2, f))

def fakerecord():
    return {'awb': faker.numerify('######'),  # random number eg:235533
            'destination_city': faker.city(),  # random cities
            'product': 'random_product',  # different products
            'product_category': 'random_category',  # different categories
            'origin_city': faker.city(),  # random metro cities
            'logistics_provider_id': faker.numerify('##'),  # id's eg:1,20,28,27
            'dispatch_date': date_between('mar01-2015', 'mar15-2015'),  # datetime between mar01-2015 to mar15-2015
            'final_delivery_status': random.choice(status),  # created,delivered,returned
            'actual_delivery_date': date_between('mar16-2015', 'mar30-2015'),  # datetime between mar16-2015 to mar30-2015
            'promised_delivery_date': date_between('mar25-2015', 'apr06-2015'),  # datetime between mar25-2015 to Apr6-2015
            }

example_dummy_data = pandas.DataFrame([fakerecord() for _ in range(1000)])

Answer 2

Found how to generate random numbers but not with in the ranges and cities i want. So Please help me how to create the dummy data like i have mentioned with 10000 rows with in the ranges i have mentioned.

Random Range:

from random import randint

xs = randint(0, 1000)  # random int between 0 and 1000

Random Selection:

from random import choice

cities = ["Brisbane", "Sydney", "Melbourne"]

random_ciy = choice(cities)  # A randomly selected city from cities

Random Date: (Thanks to Generate a random date between two other dates)

from random import randrange
from datetime import timedelta

def random_date(start, end):
    """Return a random date between two datetime objects start and end"""

    delta = end - start
    int_delta = (delta.days * 24 * 60 * 60) + delta.seconds
    random_second = randrange(int_delta)

    return start + timedelta(seconds=random_second)

Output:

>>> random_date(datetime(2015, 06, 1), datetime(2015, 9, 1))
datetime.datetime(2015, 7, 19, 11, 59, 46)

See:

• random.randint()
• random.choice()
• random.randrange()
• datetime.datetime
• datetime.timedelta

The rest if up to you how you construct your dataset