Webshere Commerce - Data Cache (Part 3) - wc-server.xml

Populating data in CacheIVL: <WCD>\schema\db2\wcs.cacheivl.trigger.sql will do the job of copulating cache table when ever there is a add,delete and update.

Configure wc-server.xml:
Ensure that you have the following element in wc-server.xml under instanceProperties element

<CrossTransactionCache
            clearCacheOnMaxInvalidationIdsPerTransaction="false"
            clearUserDataCacheOnLogoff="false" clearUserOnLogoff="true"
            commandCaching="default" defaultResultSizeThreshold="8"
            enabled="true" maxInactivityTime="86400"
            maxInvalidationIdsPerTransaction="100000"
            maxTimeToLive="172800" reduceInvalidationIds="false" reduceMemory="false">
            <WCSystemDistributedMapCache enabled="true"
                maxInactivityTime="86400" maxTimeToLive="172800" reduceMemory="false"/>
            <com.ibm.commerce.user.objsrc.OrganizationCache enabled="true"/>
            <com.ibm.commerce.taxation.objsrc.CountryCache enabled="true"/>
            <com.ibm.commerce.taxation.objsrc.StateProvinceCache enabled="true"/>
            <com.ibm.commerce.user.objsrc.MemberGroupCache enabled="true"/>
            <com.ibm.commerce.user.objsrc.RoleCache enabled="true"/>
            <com.ibm.commerce.fulfillment.objsrc.ShippingModeCache enabled="true"/>

            <com.ibm.commerce.fulfillment.objsrc.ShippingModeDescriptionCache enabled="true"/>
            <com.ibm.commerce.common.objsrc.StoreEntityDescriptionCache enabled="true"/>
            <com.ibm.commerce.common.objsrc.CurrencyCache enabled="true"/>
            <com.ibm.commerce.common.objsrc.CurrencyDescriptionCache enabled="true"/>
            <com.ibm.commerce.common.objsrc.LanguageCache enabled="true"/>
            <com.ibm.commerce.common.objsrc.LanguageDescriptionCache enabled="true"/>
            <com.ibm.commerce.catalog.objsrc.CatalogCache enabled="true"/>
            <com.ibm.commerce.catalog.objsrc.CatalogDescriptionCache enabled="true"/>
            <com.ibm.commerce.common.objsrc.SupportedLanguageCache enabled="true"/>
            <com.ibm.commerce.fulfillment.objsrc.FulfillmentCenterCache enabled="true"/>

            <com.ibm.commerce.attachment.beansrc.AttachmentRelationUsageCache enabled="true"/>

            <com.ibm.commerce.attachment.beansrc.AttachmentRelationUsageDescriptionCache enabled="true"/>

            <com.ibm.commerce.infrastructure.facade.server.commands.CreateAttachmentUsageConfigurationCmdImpl enabled="true"/>
            <com.ibm.commerce.user.beansrc.PolicyAccountCache enabled="true"/>
            <com.ibm.commerce.user.beansrc.PolicyAccountLockoutCache enabled="true"/>
            <com.ibm.commerce.user.beansrc.PolicyPasswordCache enabled="true"/>
            <WCSessionDistributedMapCache enabled="true"
                maxInactivityTime="900" maxTimeToLive="172800" reduceMemory="false"/>

            <com.ibm.commerce.component.contextservice.commands.ContextDataSerValueCacheCmdImpl enabled="true"/>
            <WCContractDistributedMapCache enabled="true"
                maxInactivityTime="86400" maxTimeToLive="172800" reduceMemory="false"/>
            <com.ibm.commerce.contract.beansrc.BusinessPolicyCache enabled="true"/>
            <com.ibm.commerce.contract.beansrc.PolicyDescriptionCache enabled="true"/>
            <WCPromotionDistributedMapCache enabled="true"
                maxInactivityTime="86400" maxTimeToLive="172800" reduceMemory="false"/>
            <com.ibm.commerce.fulfillment.objsrc.CalculationCodeCache enabled="true"/>

            <com.ibm.commerce.fulfillment.objsrc.CalculationCodeDescriptionCache enabled="true"/>

            <com.ibm.commerce.tools.epromotion.objimpl.CalculationCodePromotionCache enabled="true"/>

            <com.ibm.commerce.marketing.promotion.AbstractPromotionSessionBeanPersistenceManager enabled="true"/>

            <com.ibm.commerce.marketing.promotion.group.PromotionGroupSessionBeanPersistenceManager enabled="true"/>

            <com.ibm.commerce.marketing.promotion.policy.PromotionPolicySessionBeanPersistenceManager enabled="true"/>

            <com.ibm.commerce.marketing.promotion.runtime.ShopcartDrivenAgendaBuilder.SHOPCART enabled="true"/>

            <com.ibm.commerce.marketing.promotion.runtime.ShopcartDrivenAgendaBuilder enabled="true"/>
            <WCMarketingDistributedMapCache enabled="true"
                maxInactivityTime="86400" maxTimeToLive="172800" reduceMemory="false"/>

            <com.ibm.commerce.tools.campaigns.CampaignInitiativeEvaluateCmdImpl enabled="true"/>

            <com.ibm.commerce.marketing.commands.elements.DisplayMerchandisingAssociationActionTaskCmdImpl.MASSOCCECE enabled="true"/>

            <com.ibm.commerce.marketing.commands.elements.DisplayMerchandisingAssociationActionTaskCmdImpl enabled="true"/>
            <com.ibm.commerce.marketing.dialog.util.MarketingUtilCatalog enabled="true"/>
            <WCUserDistributedMapCache enabled="true"
                maxInactivityTime="900" maxTimeToLive="172800" reduceMemory="false"/>

            <com.ibm.commerce.membergroup.commands.ListMemberGroupsForUserCmdImpl enabled="true"/>
            <com.ibm.commerce.user.objsrc.DemographicsCache enabled="true"/>
            <com.ibm.commerce.dynacache.commands.MemberGroupsCache enabled="true"/>
            <com.ibm.commerce.user.objsrc.MemberGroupMemberCache enabled="true"/>
            <com.ibm.commerce.user.objsrc.MemberRelationshipsCache enabled="true"/>

            <com.ibm.commerce.user.beansrc.MemberRelationshipsExtendedCache enabled="true"/>
            <com.ibm.commerce.user.objsrc.MemberRoleCache enabled="true"/>
            <com.ibm.commerce.user.objsrc.MemberCache enabled="true"/>
            <com.ibm.commerce.user.objsrc.UserCache enabled="true"/>
            <com.ibm.commerce.user.objsrc.UserRegistryCache enabled="true"/>

            <com.ibm.commerce.accesscontrol.policymanager.CommandLevelAuthorizationCache enabled="true"/>

            <com.ibm.commerce.accesscontrol.policymanager.ParentOrganizationCache enabled="true"/>

            <com.ibm.commerce.marketing.promotion.runtime.PromotionArgumentSessionBeanPersistenceManager enabled="true"/>
            <WCCatalogGroupDistributedMapCache enabled="true"
                maxInactivityTime="86400" maxTimeToLive="172800" reduceMemory="false"/>
            <com.ibm.commerce.catalog.objsrc.CatalogGroupCache enabled="true"/>

            <com.ibm.commerce.catalog.objsrc.CatalogGroupDescriptionCache enabled="true"/>
            <com.ibm.commerce.catalog.beansrc.CatalogGroupRelationCache enabled="true"/>
            <WCCatalogEntryDistributedMapCache enabled="true"
                maxInactivityTime="86400" maxTimeToLive="172800" reduceMemory="false"/>

            <com.ibm.commerce.fulfillment.objsrc.CatalogEntryShippingCache enabled="true"/>
            <com.ibm.commerce.catalog.objsrc.CatalogEntryCache enabled="true"/>

            <com.ibm.commerce.catalog.objsrc.CatalogEntryDescriptionCache enabled="true"/>
            <com.ibm.commerce.catalog.objsrc.ItemCache enabled="true"/>
            <com.ibm.commerce.catalog.objsrc.ProductCache enabled="true"/>
            <com.ibm.commerce.catalog.util.CatalogHelperCache enabled="true"/>

            <com.ibm.commerce.catalog.beansrc.CatalogGroupCatalogEntryRelationCache enabled="true"/>
            <WCPriceDistributedMapCache enabled="true"
                maxInactivityTime="86400" maxTimeToLive="172800" reduceMemory="false"/>
            <com.ibm.commerce.catalog.beansrc.ListPriceCache enabled="true"/>
            <com.ibm.commerce.order.objsrc.OfferCache enabled="true"/>
            <com.ibm.commerce.order.objsrc.OfferPriceCache enabled="true"/>
            <com.ibm.commerce.price.commands.RetrievePricesCmdImpl enabled="true"/>
        </CrossTransactionCache>

After configuring distirbutedMap on server. You should see following logging statements in sysouts

ResourceMgrIm I   WSVR0049I: Binding WCUserDistributedMapCache as services/cache/WCUserDistributedMapCache
ResourceMgrIm I   WSVR0049I: Binding WCSessionDistributedMapCache as services/cache/WCSessionDistributedMapCache
ResourceMgrIm I   WSVR0049I: Binding WCCatalogEntryDistributedMapCache as services/cache/WCCatalogEntryDistributedMapCache
ResourceMgrIm I   WSVR0049I: Binding WCCatalogGroupDistributedMapCache as services/cache/WCCatalogGroupDistributedMapCache

 This means all the configured distirbutedMaps are active now.


This will ensure all the settings regarding the data cache is completed. Still i have to find if WC is considering cacheivl? is the invalidation schedule job provided by the IBM is working? why i ask this question is simple. Trigger ran twice for catalog table and there is double data in cacheivl. when i ran invalidation job. data was not removed from cacheivl. If any one know more on this please share.

Webshere Commerce - Data Cache (Part 2) - Enabling and configuring distributedMap

 There are two ways to configure data cache

• By creating cacheinstance.properties
• By configuring on application server.  This is discussed below
  

  Step By Step setup:

• Step 1: Click on Object Cache instances
  

• Step 2: Click New

• Step 3: Get the distributed Map you want configure (here)
• Step 4: Here i am setting properties for a single JVM . JNDI has to be services/cache/distributedMapName ( you will get that from Step 3)
  

Websphere Commerce Data Cache (Part 1) - Distribution Maps

 In Data Cache we have to know what distribution mapping has to be configured for caching.We cover how to configure in a while. Before configuring we need to know what are the distribution maps available.

WCContractDistributedMapCache, WCSystemDistributedMapCache

Cache any system data like Organization, Country, StateProvince,MemberGroup, Role, ShippingMode, Currency,Language, BusinessPolicy

WCPromotionDistributedMapCache, WCMarketingDistributedMapCache

These distribution maps will cache marketing and promotional data like CalculationCode, Promotion, PromotionGroup, PromotionPolicy, MerchandisingAssociation

WCCatalogGroupDistributedMapCache

This will cache catalog Group related data like CatalogGroup, CatalogGroupRelation

WCCatalogEntryDistributedMapCache, WCPriceDistributedMapCache

Catalog Entry related data will be cached for this distribution Map like Item, Product, Shipping, Price.

WCUserDistributedMapCache, WCSessionDistributedMapCache

User and session related data like user related and session related data

WCFlexFlowDistributedMapCache,WCSearchxxxDistributedMapCache,WCSEOxxxDistributedMapCache

Some Essential data.

Note: Command caching is enabled using cachespec.xml

How to configure distributedMap data cache on server? Click here

Reference:

• For list of classes included in each distribution map (Distribution Map - Classes)
• IBM Support document (IBM Support Document)

Caching intro

Well .. before i started going through the websphere commerce caching features. I was under assumption only JSPs, Static content only can be cached for easy access.

IBM Websphere commerce cache almost everything like JSP, Sevlets, Commerce commands.
What are other features IBM WC provides for caching?

• IBM WC verifies if content of cache is valid . And this validation is run on rule based, time based , group based(What is this? got to find out)  and programmatic.
• If you have multiple servers it can replicate caching on all the servers.
• When you are restarting server large chunk of cache will be preserved to reused after restart. ( this is really news to me. Most of the time we restart server to ensure there is no cache is present on server. IBM WC does a smart saving of cache) So next time if you are still seeing old data after restart remember to clear cache using cache monitor.

Now question is how are we going to enable above features and where. Here is the answer cachespec.xml

Based on type of content some content of data can be cached at browser level , If data is more secured then cache data at server level ( more secure) . IBM WC can cache full page or partial page based on what you define.

I will cover Cachespec.xml and how to utilize better caching.

Returns debugging:

select * from RMA where RMA_ID=11502;
select * from RMAITEM where RMAITEM_ID=11502;
select * from RMAIDNYRSN where RMAITEM_ID=11502;
select * from RTNDNYDESC where RTNDNYRSN_ID=-3 and LANGUAGE_ID=-1;
select * from RTNDNYRSN;

Orders that are eligible for Release:

SELECT DISTINCT ORDERS.ORDERS_ID FROM  ORDERITEMS, ORDERS WHERE ORDERS.STOREENT_ID=10151
AND ORDERS.ORDERS_ID = ORDERITEMS.ORDERS_ID  AND ORDERS.STATUS IN ('C', 'M', 'B', 'L')  AND 
( ORDERS.SHIPASCOMPLETE = 'Y' AND NOT EXISTS (SELECT ORDERITEMS_ID FROM ORDERITEMS WHERE ORDERS_ID = ORDERS.ORDERS_ID AND ( INVENTORYSTATUS NOT IN('ALLC','FUL')  )))
AND NOT EXISTS (SELECT ORDERITEMS_ID FROM ORDERITEMS WHERE ORDERS_ID = ORDERS.ORDERS_ID AND (  ORDERITEMS.ADDRESS_ID IS NULL ));

Configuring inbound messages

When I am new to WC environment. My first challenging task was to provide inbound message to our SAP team.
Given the fact that i am not new to integration, I know exactly what has to be done in typical J2EE environment but not in WC environment.

In typical J2EE environment, First i will design what should be the XML format, Then configuration MQ and many more based on the requirement. But in WC XML format is available for most of the requirements.

WC provide XML format for following functionalities.

1. Create/Update Customer
2. Update order status
3. Update product inventory
4. Create pick batch
5. Inquire PickpackList details
6. Inventory Reciept
7. Expected inventory record
8. Shipment confirmation
9. Price and availability
10. Shopping cart transfer
11. Batch availability

All the files has to adhere to the definitions given by WC. To know in details

wc-server.xml:

In wc-server.xml search for "Messaging" you will see

<Messaging
            EcInboundMessageDtdFiles="NCCommon.mod, NCCustomer_10.mod, Create_NC_Customer_10.dtd, Update_NC_Customer_10.dtd, Update_NC_OrderStatus_10.dtd, Update_NC_ProductInventory_10.dtd, Update_NC_ProductPrice_10.dtd, Create_WCS_Customer_20.dtd, Create_WCS_Customer_30.dtd, Update_WCS_Customer_20.dtd, Update_WCS_Customer_30.dtd, Update_WCS_OrderStatus_20.dtd, Update_WCS_ProductPrice_20.dtd, Inquire_WCS_PickPackListDetail_10.dtd, Create_WCS_PickBatch_10.dtd, Create_WCS_ExpectedInventoryRecord_10.dtd, Create_WCS_InventoryReceipt_10.dtd, Update_WCS_InventoryReceipt_10.dtd, Create_WCS_ShipmentConfirmation_10.dtd, Create_WCS_ShipmentConfirmation_20.dtd, Update_WCS_ProductInventory_20.dtd, Request_WCS_BE_ProductInventory_10.dtd, Update_WCS_OrderStatus_30.dtd, Update_WCS_PriceAndAvailability_10.dtd, Update_WCS_ShoppingCartTransfer_10.dtd, Update_WCS_BatchAvailability_10.dtd"
            EcInboundMessageDtdPath="messaging"
            EcMimePropFile="lang_mime.data"
            EcSystemTemplateFile="sys_template.xml"
            EcTemplatePath="messaging"
            EcUserTemplateFile="user_template.xml" XMLWebControllerUserId="wcsadmin"/>

 EcInboundMessageDtdFiles gives information on what all definitions are available in WC.
 EcInboundMessageDtdPath gives where these files are present
 EcSystemTemplateFile gives the mapping between dtd file and parameters we can use in the code.
 EcUserTemplateFile : If you want to update any parameter in sys_template.xml , Copy entire template to user_template.xml and modify in this file. Do not modify anything in sys_template

Lets take and example and see understand what is happening.

Consider your procurement team found a particular order has to be returned/refund. Procurement has to send message to WC.

How should be the structure of xml? Open Update_WCS_OrderStatus_20.dtd . This file give the definition of how should the xml looks like.

After procurement team creates the xml and send to WC. WC reads the understands the xml. But how would we access the values defined in the xml? for that
Open sys_template.xml and search for Update_WCS_orderstatus and consider the version in TemplateDocument/DocType version attribute is has to be 2.0

Now you understand name of the dtd and name inside the sys_template.xml are related for understanding purpose.

Lets go little depth to understand sys_template:

<TemplateDocument>
        <DocumentType version='2.0'>Update_WCS_OrderStatus</DocumentType>
        <StartElement>Update_WCS_OrderStatus</StartElement>
        <TemplateTagName>OrderStatus20Map</TemplateTagName>
        <CommandMapping>
            <Command CommandName='OrderConfirmStatus' Condition='OrderConfirmCommand' />
            <Command CommandName='OrderShippingStatus' Condition='OrderShippingCommand' />
            <Command CommandName='OrderInvoiceStatus' Condition='OrderInvoiceCommand' />
            <Command CommandName='OrderStatus' Condition='OrderStatusCommand' />
        </CommandMapping>
</TemplateDocument>

Notice TemplateTagName element in above example. This element states that refer to
<TemplateTag name='OrderStatus30Map'> in sys_template for relation between DTD and WC parameters.

Lets take a tag
<Tag XPath='DataArea/OrderStatus/OrderStatusItem/TotalPriceInfo/TotalNetPrice' Field='priceTotal' />

This tag states value under Xpath DataArea/OrderStatus/OrderStatusItem/TotalPriceInfo/TotalNetPrice  is present in priceTotal parameter.

Lets move to CommandMapping element:

 Below tag defines, If xml contains XPath = DataArea/OrderStatus then OrderStatusCommand condition matches with commandName OrderStatus

 <Tag XPath='DataArea/OrderStatus' Field='OrderStatusCommand' FieldInfo='COMMAND' />

 To know what command will be triggered for OrderStatus , Search in struts-config.xml for "/OrderStatus", by this you will know what command is called.

For customizing extend the command the command and overwrite

public  boolean doProcess(TypedProperty typedproperty)

typedproperty contains the properties that are defined in sys_template.xml

To find outbound messages OOTB provides click here

Splitting Order Items for Partial Shipping

If you are depending on external system for updating shipped quantity and you had to split order items instead of orders based on the quantity shipped for that order item.

We had a scenario where our external system is SAP and WC (Websphere Commerce) has to act based on the information SAP is sending to WC. There can be Partial shipping, Partial Cancellation of and order item.

IBM WC 7 is not supporting partial shipping by default. So we have to handle this scenario explicitly.

When SAP send message stating quantity of order item shipped. We have to split the order items through code. Here is the approach for splitting the order item.

 void createPartialOrderItem(OrderItemAccessBean originalOI,String quantityShipped,Long orderId) throws ECException, RemoteException, CreateException, FinderException, NamingException {
            CSROrderItemAddCmd addCmd = (CSROrderItemAddCmd) CommandFactory.createCommand(CSROrderItemAddCmd.NAME, getStoreId());
           
              Hashtable hashtable4 = new Hashtable();
              hashtable4.put("orderId", orderId.toString());
              hashtable4.put("customerId", "-1000"); // Note only few users have authority to create order item, Here -1000 is admin id
              Hashtable hashtable3 = new Hashtable();
              hashtable3.put("order", hashtable4);
              Hashtable hashtable2= new Hashtable();
              Hashtable hashtable5= new Hashtable();
              hashtable5.put("catentryId", originalOI.getCatalogEntryId());
              hashtable5.put("quantity",quantity );
              hashtable5.put("partNumber",originalOI.getPartNumber());
              hashtable5.put("tradingId",originalOI.getContractId() );
              hashtable5.put("shipModeId", originalOI.getShippingModeId());
              hashtable5.put("shipAddrId", originalOI.getAddressId());
              Vector vector3= new Vector();
              vector3.add(hashtable5);
              hashtable3.put("orderItem", vector3);
              hashtable2.put("XML",hashtable3);
             
              CommandContext contexttemp21 = (CommandContext)getCommandContext().clone();
              contexttemp21.setStoreId(storeIDInteger);
              contexttemp21.setStore(getCommandContext().getStore(storeIDInteger));
              addCmd.setCommandContext(contexttemp21);
              Hashtable hassmap= new Hashtable();
              hassmap.put("xml_string",hashtable2);
             
              Vector vector2= new Vector();
              vector2.add(hashtable2);
             
              requestProperties.put("EC_XMLObject", vector2);
              addCmd.setDefaultProperties(requestProperties);
              addCmd.setRequestProperties(requestProperties);
              addCmd.execute();
        }

The user does not have the authority to run this command "com.ibm.commerce.user.beans.OrganizationDataBean".

Issue:

javax.servlet.ServletException: com.ibm.commerce.exception.ECApplicationException: The user does not have the authority to run this command "com.ibm.commerce.user.beans.OrganizationDataBean".


How to Replicate:
There could be several scenarios to replicate this. This is one scenario to replicate.

1. Register user  to an organization say Organization X.
2. In Organization console give privileges to user as buyer for organization Y.
3. Now log on to your storefront and change the role of user with which he is buying the products ( In this case Organization Y). This step may not be present for all the sites. Point to note is User has to place an order as buyer for organization Y.
4. Now in any of the jsp/class try to access OrganizationDataBean by using the organization user registered with ( In this example: Organization X)

For example,
I was using this below code

UserRegistrationDataBean buyurdb=new UserRegistrationDataBean();
    buyurdb.setUserId(requisitionerId);
    com.ibm.commerce.beans.DataBeanManager.activate(buyurdb, request);
 
    OrganizationDataBean buyorgdb = new OrganizationDataBean();
    buyorgdb.setInitKey_MemberId(buyurdb.getOrganizationId());
     com.ibm.commerce.beans.DataBeanManager.activate(buyorgdb, request);

Cause:
If you notice USERREG table has the data stating this user is registered to Organization X, But not the role with which he is buying in the storefront.

Fix: 
My objective was to get the organization information with which user placed the order. When user place the order , organization id will be stored in ORDERS table. Get the organization id from ORDERS table instead of UserRegistationDataBean.

 OrganizationDataBean buyorgdb = new OrganizationDataBean();
    buyorgdb.setInitKey_MemberId(buyurdb.getOrganizationId()); --Member id has to get from ORDERS table
     com.ibm.commerce.beans.DataBeanManager.activate(buyorgdb, request);