blog oofdev

Sorting dataframe by columns which contains nan values. DataFrame has "sort_values()" method can take an another parameter called "na_position".Using this parameter the rows containing nan values can be pushed to either top or bottom Creating a new dataframe with dictionary  # importing pandas import pandas as pd import numpy as np # animal_data dictionary animal_data = { "Name": ["Cat", "Dog", "Cow"], "Speed": [15, 12, 10], "Sound": ["Meow", "Woof", "Mooo"], "Rank": [1, 5, 3], "Jumping_height": [20, 10, np.NaN], } # creating a dataframe using the animal_data dictionary animal_df = pd.DataFrame(animal_data) # printing animal_df print("animal_df \n", animal_df) animal_df Name Speed Sound Rank Jumping_height 0 Cat 15 Meow 1 20.0 1 Dog 12 Woof 5 10.0 2 Cow 10 Mooo ...

Sorting pandas dataframe by single or multiple columns. DataFrame has "sort_values()" method which can be used to sort the dataframe based single or multiple columns , control sorting flow and choose ascending or descending order. Creating a new dataframe with dictionary  # importing pandas import pandas as pd import numpy as np # animal_data dictionary animal_data = { "Name": ["Cat", "Dog", "Cow"], "Speed": [15, 12, 10], "Sound": ["Meow", "Woof", "Mooo"], "Rank": [1, 5, 3], "Jumping_height": [20, 10, np.NaN], } # creating a dataframe using the animal_data dictionary animal_df = pd.DataFrame(animal_data) # printing animal_df print("animal_df \n", animal_df) animal_df Name Speed Sound Rank Jumping_height 0 Cat 15 Meow 1 20.0 1 Dog 12 Woof 5 10.0 2 Cow 10 Mooo 3 ...

Find and replace string values in a column - pandas dataframe String type has str.replace() method which can used for finding and replacing values.We are required to chain this method string type values and pass parameters for value to be searched and new replacing value. Based on the dataset one might be required to run search and replace on columns with mixed datatypes ie. int ,etc. To handle this assert type as string and then chain required string methods Creating a new dataframe with dictionary  # importing pandas import pandas as pd # animal_sp_char_df - with special characters animal_data_with_sp_char = { "Name": ["Cat", "Dog", "Cow", "Tiger", "Goat", "Snake"], "Sound": ["#Meow###", "Wo##of", "Mo#oo", "Rwaar###", "##Baaa", "Skkk##sss"], "Mixed": [123, "#13", "53###", 321, "###456", ...

Selecting every nth row from the dataframe. We can select every nth row item from the pandas dataframe by using ".iloc" method. It has the slicing features and stepping features similar to list slicing. iloc is index based and starts from zero Creating a new dataframe with dictionary  # importing pandas import pandas as pd # animal_data_ animal_data_ext = { "Name": ["Cat", "Dog", "Cow","Tiger","Goat","Snake"], "Sound": ["Meow", "Woof", "Mooo","Rwaar","Baaa","Skkksss"], } #creating a dataframe using the animal_data_ext dictionary animal_ext_df = pd.DataFrame(animal_data_ext) #printing animal_ext_df print("animal_ext_df \n", animal_ext_df) animal_ext_df Name Sound 0 Cat Meow 1 Dog Woof 2 Cow Mooo 3 Tiger Rwaar 4 Goat Baaa 5 Snake Skkksss Selecting every nth row (includi...

Lowercase / uppercase all column cell contents. String type has methods for lower-casing and upper-casing. A column is selected from dataframe and str (string) operations are made on it. But this would throw errors for int data types. We can either skip over those columns using conditionals or use "astype" assert it as a string and then operate string methods on it without any error.Based on requirement one might choose to either skip over or handle all columns with mixed type contents . Creating a new dataframe with dictionary (which has alpha numeric cell contents) # importing pandas import pandas as pd # animal_data_with_alpha_nums dictionary animal_data_with_alpha_nums = { "Name": ["Cat", "Dog", "Cow"], "Speed": [15, 12, 10], "Sound": ["Meow", "Woof", "Mooo"], "Alpha_Num":[123,"aBc123XyZ","123xYz"] } #creating a dataframe using the a...

Generating column unique values from all columns from a dataframe and storing it into python dictionary. Dataframe has "unique" method which returns the required unique values.  Dataframe "columns" property will also be used to get the column names which will iterated over for the purpose. Creating a new dataframe with dictionary (which has duplicate data) # importing pandas import pandas as pd # animal_data_with_duplicates dictionary animal_data_with_duplicates = { "Name": ["Cat", "Dog", "Cow","Tiger","Cat"], "Speed": [15, 12, 10,20,15], "Sound": ["Meow", "Woof", "Mooo","Roar","Meow"], } #creating a dataframe using the animal_data_with_duplicates dictionary animal_with_duplicates_df = pd.DataFrame(animal_data_with_duplicates) #printing animal_with_duplicates_df print("animal_with_duplicates_df \n", animal_with_...

Generating column unique values from a dataframe. Dataframe has "unique" method which returns the required unique values. Creating a new dataframe with dictionary (which has duplicate data) # importing pandas import pandas as pd # animal_data_with_duplicates dictionary animal_data_with_duplicates = { "Name": ["Cat", "Dog", "Cow","Tiger","Cat"], "Speed": [15, 12, 10,20,15], "Sound": ["Meow", "Woof", "Mooo","Roar","Meow"], } #creating a dataframe using the animal_data_with_duplicates dictionary animal_with_duplicates_df = pd.DataFrame(animal_data_with_duplicates) #printing animal_with_duplicates_df print("animal_with_duplicates_df \n", animal_with_duplicates_df) animal_with_duplicates_df Name Speed Sound 0 Cat 15 Meow 1 Dog 12 Woof 2 Cow 10 Mooo 3 Tiger 20 Roar 4 Cat 15 Meow #g...

Generating index list from a dataframe. Dataframe object has "index" method which returns the required data object. This can be converted into list. Index list can be used for counting of current number of rows , for using ".iloc" , shuffling etc. Creating a new dataframe with data # importing pandas import pandas as pd # animal_data dictionary animal_data = { "Name": ["Cat", "Dog", "Cow"], "Speed": [15, 12, 10], "Sound": ["Meow", "Woof", "Mooo"], } #creating a dataframe using the animal_data dictionary animal_df = pd.DataFrame(animal_data) #printing animal_df print("animal_df \n", animal_df) animal_df Name Speed Sound 0 Cat 15 Meow 1 Dog 12 Woof 2 Cow 10 Mooo # create a list containing values indices_as_list = list(animal_df.index) print("indices_as_list \n", indices_as_list) print("\n length of indices_as_...

Generating column names list from a dataframe. Dataframe object has "columns" method which returns the required data object. This can be converted into list.  Creating a new dataframe with data # importing pandas import pandas as pd # animal_data dictionary animal_data = { "Name": ["Cat", "Dog", "Cow"], "Speed": [15, 12, 10], "Sound": ["Meow", "Woof", "Mooo"], } #creating a dataframe using the animal_data dictionary animal_df = pd.DataFrame(animal_data) #printing animal_df print("animal_df \n", animal_df) animal_df Name Speed Sound 0 Cat 15 Meow 1 Dog 12 Woof 2 Cow 10 Mooo # create a list containing values column_names_as_list = list(animal_df.columns) print("column_names_as_list \n", column_names_as_list) print("\n length of column_names_as_list \n", len(column_names_as_list)) #or directly use the values a...

Creating a pandas dataframe with dictionary data. A dictionary containing lists as values can directly passed into pd.Dataframe method to create a new dataframe using the passed data. The keys in the dictionary will be used as the column headers and values in the list will be used as row values. # importing pandas import pandas as pd # animal_data dictionary animal_data = { "Name": ["Cat", "Dog", "Cow"], "Speed": [15, 12, 10], "Sound": ["Meow", "Woof", "Mooo"], } #creating a dataframe using the animal_data dictionary animal_df = pd.DataFrame(animal_data) #printing animal_df print("animal_df \n", animal_df) animal_df Name Speed Sound 0 Cat 15 Meow 1 Dog 12 Woof 2 Cow 10 Mooo

TypeDoc makes building documentations very easy. It can fit into most codebases with minimal setup. TypeDoc extracts & builds documentation from typescript types , interfaces,.. and also makes use javascript doc comments. Install dependencies: npm i typedoc -d yarn add typedoc -d //installing as dev dependency After installation , create a "typedoc.json" in the root path. Find more resources here  https://typedoc.org/options/configuration/ Configuration file I have customised  https://github.com/808vita/nextjs-blog-with-auth/blob/deploy/typedoc.json Repository GitHub link :  Full repository github - 808vita -oofdev

Have you tried opening a very big csv file with excel?  I tried to open a csv file with 5million rows of data using Microsoft Excel : 1,048,576 rows limit . 1 million row limit ,I was aware of this. Not all the data was loaded , even got a notification modal stating this. 32,767 character per cell , this I was not aware of. After opening a file which exceed this limit , new rows were displayed and looked like a mess. But the file was properly formatted when opened with notepad. This one is obvious but formulas and filters were very slow (given the size of the data ,expected). Why I was required to open a file with over 5million rows in the first place ? I was actively trying to learn machine learning and tried to build dataset for supervised learning. I wanted to open the file to mark classes and values - for training classification and regression models. Workarounds I did include : filtered and removed currently unused rows . This cut the size by almost half. split the files into ...

 Python ast.literal_eval()  Is a very useful method which can evaluate a string contents for python datatypes . If the string happens to have valid datatype inside the string it will initialised . One use case in which I personally used it was to convert strings values stored csv files . The rows contained "list" as string and wanted to run operations on it. These must be converted into list format first ; ast.literal_eval()  achieves that ! #python code import ast #string containing a list stringed_list= "[1,2,3]" #converting stringed list into python list converted_list = ast.ast.literal_eval(string_list) print(type(stringed_list)) # 'str' print(type(converted_list)) # 'list' Even though this is a really convenient way to convert stringed list back into python list ,  it is slow. This works great for moderately small csv data files in which we can store scaled parameter lists , etc as string .  When size of the file increases so does ...

 Sample images generated in stable-diffusion-webui

How to set Clip skip slider quick setting -->Click on Settings tab -->select User interface section

Stable-Diffusion-Webui Lora usage  Currently running 1.5.2 Lora can be used directly within webui without requiring any additional installation after setup. Where to place Lora files to use it within stable-diffusion-webui : (your-file-path) \stable-diffusion-webui\stable-diffusion-webui\models\Lora (make sure to download the compatible Lora files. SDv1.5 trained Lora will not run on SDv2+ .) How to activate Lora? --> Click on show/ hide extra networks -->Tab over to Lora . All available Lora files will be displayed . -->You can activate / use the lora prompt by simply clicking on the required Lora . Adjust the intensity to required level . Scale is 0 to 2. More Settings for Lora files and check details  -->click edit (tools symbol) --> Details about the Lora will be displayed including the related activation texts . --> Quick activation can be set here . -Auto generate prompts for copy paste , it is randomly generated  -Using the slider you can set...

 After generating the images the files will be stored to dated folders based on type of generation used. Located here : (your-file-path) \stable-diffusion-webui\stable-diffusion-webui\outputs Images generated using: Extras Tab -  will be stored to extras-images . Img2img - separate images will be stored to img2img-images . Grid images will be stored to img2img-grid . The same paths are used for inpaint generation. Txt2img -  separate images will be stored to txt2img-images . Grid images will be stored to  txt2img -grid .

webui-user.bat Various configuration for stable-diffusion-webui webui-user.bat file. Default file configuration The configuration available by default after cloning stable-diffusion-webui inside the webui-user.bat file. @echo off set PYTHON= set GIT= set VENV_DIR= set COMMANDLINE_ARGS= call webui.bat File configuration below 8gb gpu and 16xx series Use the below code if your gpu is under 8gb and of 16xx nvidia series. With this half precision errors will get resolved and more memory will be available for rendering. @echo off set PYTHON= set GIT= set VENV_DIR= set COMMANDLINE_ARGS= --no-half-vae --medvram call webui.bat File configuration for openpaint extension  Add --api flag to the args . This can also be clubbed with other args. This flag has to be set to use the openpaint withing the stable-diffusion-webui  @echo off set PYTHON= set GIT= set VENV_DIR= set COMMANDLINE_ARGS= --api call webui.bat File configuration for openpaint extensio...

  Both getStaticProps and getServerSideProps usually returns props but both functions can also return "redirect" object. Redirect object is used to redirect the user to a different url.  Invalid Redirect getStaticProps/getServerSideProps error occurs when redirect object that was returned did not have correct structure . To fix the issue make sure the redirect object holds proper values in correct shape  { destination: string, permanent: boolean } . destination requires a url path in string example "/" . permanent requries a boolean value. You can instead use statusCode can also be used but not both together . statusCode requires a number values . This would represent the status code . export async function getStaticProps() { const data = null; // since data is to null if condition will be validated & redirect will occur if (!data) { return { redirect: { destination: "/", permanent: false, // statusCode: 3...

Type error: 'GetServerSideProps' refers to a value, but is being used as a type here. Did you mean 'typeof GetServerSideProps' This error occurs when you use imported "GetServerSideProps" server side props function . "GetServerSideProps" is the typescript type of "getServerSideProps". To fix the error : If you are using javascript then just remove the import GetServerSideProps from "next" and then change the server side function to: export const getServersideProps = async () => {   return { props: { oof: "hello" } }; }; If codebase is typescript based then use "GetServerSideProps" as the type and modify the function to : export const getServerSideProps: GetServerSideProps = async () => {   return { props: { oof: "hello" } }; }; You might also encounter other errors when using the getServerSideProps : ` undefined` cannot be serialized as JSON. Please use `null` or omit this val...

  When a page component makes use getServerSideProps function and does not return a valid props object then this error will be thrown on to the console . To fix the error make sure to always have a return statement which returns an object {props:{oof:"oof"}} export async function getServerSideProps() { //anyValidData can be number , string , boolean , array , objects & null // but do not pass undefined // keep in mind these will be serialized (aka made into valid json ) return { props: { yourObjectKey: anyValidData , yourObjectKey2: anyValidData2 } } } You might also encounter other errors when using the getServerSideProps : ` undefined` cannot be serialized as JSON. Please use `null` or omit this value. Invalid getServerSideProps Return Value You can not use getStaticProps with getServerSideProps. To use SSG, please remove getServerSideProps

  When page uses getServerSideProps nextjs will try to server side render the page . That is getServerSideProps  executes api fetch calls to get data & feed into the page and return the page html (all this is done on the server side). getServerSideProps  gets triggered "at request time" aka when someone (browser) requests a page which has been marked as SSR . It gets triggered on each request. getStaticPaths gets triggered "at build time" aka when the next build command is used. Pages which uses getStaticPaths gets data from the motioned sources and creates static html pages. To fix issue first decide whether the current page should be a static html page ! If the answer is yes then remove the getServerSideProps and keep using getStaticProps . This will make the current page static html page. If  the answer is no then remove the getStaticProps and keep using getServerSideProps . This will make the current page to be server side rendered ...

This error was widely noticed when codebase gets upgraded to React 18 / latest versions of Nextjs. If you are using newer versions of Nextjs , then it is better to use getStaticProps instead of getInitialProps . Do not mix both together , either use getStaticProps or getInitialProps . getStaticProps - when used on page leads to static site generation (SSG) getServerSideProps - when used on page leads to server side rendering (SSR) & only available on page components . You might also encounter other errors when using nextjs: ` undefined` cannot be serialized as JSON. Please use `null` or omit this value. Your `getServerSideProps` function did not return an object. Did you forget to add a `return`? You can not use getStaticProps with getServerSideProps. To use SSG, please remove getServerSideProps

This error caused inside getServerSideProps code block when the props return values does not follow the required structure . The return value should have a "props" as key and then {"propertytKeyAsString":anyValidData }. Here: "propertytKeyAsString" - the key (name) you assign for the property. This will be de-structured to get property value inside the page component . "anyValidData" includes any valid data type that can be assigned  , it could be number , string , boolean, array , objects & null . Caution do not try to pass values of "undefined" data type , this will cause a different error . To fix Invalid getServerSideProps Return Value  Follow the required props structure and return a valid data export async function getServerSideProps() { //anyValidData can be number , string , boolean , array , objects & null // but do not pass undefined // keep in mind these will be serialized (aka made into valid json ) return { ...

 GetServerSideProps at request time fetches data from api endpoints and renders a content on the page,  a pretty neat feature.  We directly make the api call within the getServerSideProps function block which gets executed at the server side (not the client side aka browser). External api endpoints are on different sever and the way to interact & get data from them is by making api call . But what if it is an internal api endpoint? Like the "localhost:3000/api/hello" api endpoint is being called in page path "localhost:3000/blog/is-there-a-better-way" . As you can see both things reside on "localhost:3000" Nextjs documentation calls this inefficient and instead actually execute the all required operations from the getServerSideProps code block https://nextjs.org/docs/basic-features/data-fetching/get-server-side-props#getserversideprops-or-api-routes // path path -> src/pages/blog/alternative export default function InternalApiTest({ data }) { ...

  Error : Error serializing `.oof` returned from `getServerSideProps` in "/oof". Reason: `undefined` cannot be serialized as JSON. Please use `null` or omit this value. You got this error while using getServerSideProps . Everything is working but not able to pass the values to props. Few things to note before proceeding forward: this is a server side error , meaning you have problematic code executed on the server side inside getServerSideProps . current content cannot be serialized as json (aka make it into a valid json format ) .  this could be because of date object , "undefined" , etc. The above screenshot contains getServerSideProps code block which caused the error. The oof variable was set to "undefined" intentionally to cause the error but you might encounter this error while using url params or other situations . If you face this error check what data is being passed and if undefined be passed make sure to set it as "null" instead . Rea...

 GetServerSideProps(context) - NextJs example GetServerSideProps is executed in the server side but what if we want to access the resolved url , params or query attached to the url ? This is when "context" parameter of getServerSideProps is used .  Context is an object so we de-structure the items we want like the usual way. It contains several useful values: query - with this we can get the queries attached to current url . Example - "localhost:3000/?search="oof"" , using the context we can access the query ({search:"oof"}) from the server side & execute operations. (note: query object can also hold the param value if it exists.) params - get the params data if it exists on server side. The params would still follow the dynamic route set key name . For example if a dynamic route page like "pages/post/[id].js" , the param would be stored as {id:1}.(like the useRouter) resolvedUrl - stripped down version of requested url with paths ...

 GetServerSideProps Example NextJs GetServerSideProps when used in Nextjs several things happen - Code placed within this function gets run only on the serverside  Getserversideprops is run before the Page Component (note: this can used only on Page level) The page which contains this function would be Server Side Rendered (SSR - this would be marked with ssr symbol in the build production logs) Frontend does not have access this function, so business logic can be placed here. Code example: /** * * @param {*} jsonData * * This is the page component. * Remember getServerSideProps can only be used on Page level! * * */ export default async function Home ({ jsonData }) { // destructured jsonData which was passed from gerserversideprops // note this is in JSON , so make sure to parse it console.log(jsonData, "data"); // parsing the json data - with this we have a workable object (it could be any array , object, arary of objects & so on) ...