Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Most efficient way to produce a Python dictionary given a list

I'm looking for optimizations on the following problem (I've got some working code but I'm fairly sure it could be quicker and is written in a bad way). I've got a list of SKUs (between 6 and 9 digit numbers) that I'm looking up information on on Amazon. The working code is given below:

def buildDictionary2(stringy):
    x = stringy.xpath('//sellersku/text()|//product/descendant::amount[1]/text()')
    Sku_To_Price = {}
    for items in range(len(x)):
        if x[items] in myList:
            try:
                if x[items+1] not in myList:
                    Sku_To_Price[x[items]] = x[items+1]
                else:
                    Sku_To_Price[x[items]] = ''
            except:
                pass
        else:
            pass
    return Sku_To_Price

where x is a dictionary of generally alternating SKUs and prices. However, the complication arises whereby the price can't be found. In this case the list (x) goes SKU, SKU instead of SKU price.

At the moment I'm looking up in the list of SKUs (in the global variable myList) but can't help this is of time complexity O(e^n). Given I'm looking at working with something in the region of 20,000 SKUs I'd rather this wasn't the case.

Is there a way to make this less complex - the desired output is a dictionary with each SKU once (as a key) and it's corresponding price as the value (with no entry if if there is no price).

edit:

a sample of the XML being parsed

<?xml version="1.0" ?>
<GetLowestOfferListingsForSKUResponse xmlns="http://mws.amazonservices.com/schema/Products/2011-10-01">
  <GetLowestOfferListingsForSKUResult SellerSKU="X" status="Success">
    <AllOfferListingsConsidered>true</AllOfferListingsConsidered>
    <Product xmlns="http://mws.amazonservices.com/schema/Products/2011-10-01" 
             xmlns:ns2="http://mws.amazonservices.com/schema/Products/2011-10-01/default.xsd">
      <Identifiers>
        <MarketplaceASIN>
          <MarketplaceId>X</MarketplaceId>
          <ASIN>X</ASIN>
        </MarketplaceASIN>
        <SKUIdentifier>
          <MarketplaceId>X</MarketplaceId>
          <SellerId>X</SellerId>
          <SellerSKU>10065897</SellerSKU>
        </SKUIdentifier>
      </Identifiers>
      <LowestOfferListings>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>New</ItemCondition>
            <ItemSubcondition>New</ItemSubcondition>
            <FulfillmentChannel>Amazon</FulfillmentChannel>
            <ShipsDomestically>True</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>3</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>23.68</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>False</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>X</ItemCondition>
            <ItemSubcondition>X</ItemSubcondition>
            <FulfillmentChannel>Merchant</FulfillmentChannel>
            <ShipsDomestically>X</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>X</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>X</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>X</ItemCondition>
            <ItemSubcondition>X</ItemSubcondition>
            <FulfillmentChannel>X</FulfillmentChannel>
            <ShipsDomestically>X</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>X</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>X</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>X</ItemCondition>
            <ItemSubcondition>X</ItemSubcondition>
            <FulfillmentChannel>X</FulfillmentChannel>
            <ShipsDomestically>X</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>X</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>X</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>X</ItemCondition>
            <ItemSubcondition>X</ItemSubcondition>
            <FulfillmentChannel>X</FulfillmentChannel>
            <ShipsDomestically>X</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>X</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>X</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
      </LowestOfferListings>
    </Product>
  </GetLowestOfferListingsForSKUResult>
  <GetLowestOfferListingsForSKUResult SellerSKU="X" status="X">
    <AllOfferListingsConsidered>X</AllOfferListingsConsidered>
    <Product xmlns="http://mws.amazonservices.com/schema/Products/2011-10-01"
             xmlns:ns2="http://mws.amazonservices.com/schema/Products/2011-10-01/default.xsd">
      <Identifiers>
        <MarketplaceASIN>
          <MarketplaceId>X</MarketplaceId>
          <ASIN>X</ASIN>
        </MarketplaceASIN>
        <SKUIdentifier>
          <MarketplaceId>X</MarketplaceId>
          <SellerId>X</SellerId>
          <SellerSKU>9854521</SellerSKU>
        </SKUIdentifier>
      </Identifiers>
      <LowestOfferListings>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>X</ItemCondition>
            <ItemSubcondition>X</ItemSubcondition>
            <FulfillmentChannel>X</FulfillmentChannel>
            <ShipsDomestically>X</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>1</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>2.68</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>X</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>X</ItemCondition>
            <ItemSubcondition>X</ItemSubcondition>
            <FulfillmentChannel>X</FulfillmentChannel>
            <ShipsDomestically>X</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>8</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>X</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>X</ItemCondition>
            <ItemSubcondition>X</ItemSubcondition>
            <FulfillmentChannel>Merchant</FulfillmentChannel>
            <ShipsDomestically>X</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>4</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>X</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
        <LowestOfferListing>
          <Qualifiers>
            <ItemCondition>X</ItemCondition>
            <ItemSubcondition>X</ItemSubcondition>
            <FulfillmentChannel>X</FulfillmentChannel>
            <ShipsDomestically>X</ShipsDomestically>
            <ShippingTime>
              <Max>X</Max>
            </ShippingTime>
            <SellerPositiveFeedbackRating>X</SellerPositiveFeedbackRating>
          </Qualifiers>
          <NumberOfOfferListingsConsidered>1</NumberOfOfferListingsConsidered>
          <SellerFeedbackCount>X</SellerFeedbackCount>
          <Price>
            <LandedPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </LandedPrice>
            <ListingPrice>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </ListingPrice>
            <Shipping>
              <CurrencyCode>X</CurrencyCode>
              <Amount>X</Amount>
            </Shipping>
          </Price>
          <MultipleOffersAtLowestPrice>X</MultipleOffersAtLowestPrice>
        </LowestOfferListing>
      </LowestOfferListings>
    </Product>
  </GetLowestOfferListingsForSKUResult>
  <ResponseMetadata>
    <RequestId>X</RequestId>
  </ResponseMetadata>
</GetLowestOfferListingsForSKUResponse>

and myList looks like:

myList = ['10032590',
'10043503',
'10047539',
'10055404',
'10058424'...
]

Using the first answer below I'm getting the following error message:

TypeError: unhashable type: 'list'

where I believe the relevant code is:

def xml_to_dict(self, xml):
    doc = lh.fromstring(xml)
    d = {}
    for product in doc.xpath('.//product'):
        sku = product.xpath('.//sellersku/text()')
        amount = product.xpath('./descendant::amount[1]/text()')
        d[sku] = amount
    return d
like image 952
Kali_89 Avatar asked Jan 17 '13 17:01

Kali_89


1 Answers

d={}
for product in doc.xpath('.//product'):
    sku = product.xpath('.//sellersku/text()')[0]
    price = product.xpath('./descendant::amount[1]/text()')
    if price: # if theres a possibility of sku missing replace with:
              # "if price and sku"
              #
              # if you have duplicate sku's and you don't want them overwritten 
              # add "and sku not in d" check
        d[sku]= price[0]
like image 173
root Avatar answered Oct 10 '22 22:10

root