# Sliding window problem

Given a large integer array of size x, window size of n and a random number k, find smallest k numbers in every window of n elements in array. This is commonly know as sliding window problem. For example: for an array [2,3,1,5,6,4,2,5,4,3,8] k = 2 and n = 6, output should be [1,2],[1,2],[1,3][1,4][1,3][1,3]. How? see below figure.

This problem regularly features in Amazon interviews.

## Find k numbers in sliding window : thoughts

If we spit down the problem, it reduces to find k smallest elements in an array, which can easily be solve in multiple ways. All we have to take care of is moving the window and storing results for each window.

Quick sort method
First way is to use quick sort, we randomly pick a pivot and put it in right place. When pivot is at right place, all elements on the right side of pivot are greater than pivot and all elements on the left side are less than pivot. If pivot is a kth position in array, all elements on left side of pivot automatically become K smallest elements of given array. In worst case this method take O(n log n) for each window.

Using heaps
What are we interested in is k elements, what if from current window, we take out first k numbers and consider them as k smallest elements? This set of k numbers may change based value of following numbers in the window. Which way? If new number is smaller than any of the number chosen randomly, new number has to be added into the k smallest element set. However, we have only k spaces there, so someone has to move out.

If new number is less than any number in set, it must be less than maximum number in set

Given above fact, we can always swap new number with maximum of set. Now problem is how to find max in a set? This set will modified repeatedly, so we cannot just sort it once and find the max. For use cases when data is changing and we have to find max of that set, heaps are the best data structures to use. In this case we will use max heap. Max heap is kind of heap where children of root node are smaller than root node. Max heap will give us O(1) complexity to find max and O(log n) complexity to heapify on removal old max and insertion of new number.

Algorithm

1. Create a max heap with first k elements of window.
2. Scan through remaining elements in window
1. If root of max heap is less than new number, remove the root and add new element to heap
2. All elements in heap at the end of processing are k smallest numbers in window.

### Sliding window algorithm to find k smallest elements : Implementation

```#include<stdio.h>
#include<stdlib.h>
#include <math.h>

typedef struct node {
struct node * left;
struct node * right;
int data;
} heapNode;

int leftChild(int i){
return 2*i + 1;
}

int rightChild(int i){
return 2*i + 2;
}

void swapPtr(heapNode *a[], int i, int largest){
heapNode *temp = a[i];
a[i] = a[largest];
a[largest] = temp;
}
/* This function heapifies heap after removal of root
or at time of building heap from an array */
void max_heapify_ptr(heapNode *a[], int i, int len){
int largest = i;
int left, right;

left = leftChild(i);
right = rightChild(i);

if(left <= len && a[i]->data <a[left]->data){
largest = left;
}
if(right <= len && a[largest]->data < a[right]->data){
largest = right;
}
if(largest != i){
swapPtr(a, i, largest);
max_heapify_ptr(a, largest, len);
}
}

/* Building heap from given elements */
void build_max_heap_ptr(heapNode *a[], int len){
int i = len/2 +1;
for(; i>=0; i--){
max_heapify_ptr(a,i, len);
}
}

/* This function allocates node of heap */
heapNode * create_node(int data){
heapNode *node = (heapNode *)(malloc)(sizeof(heapNode));
if(node){
node->data = data;
}
return node;

}

/* This function is real implementation of
the sliding window algorithm */
void slide_window(int buffer[], int N, int K, int buffer_len){

int i =0, j =0,s;
heapNode *max_heap[K+1];
int num = K;

for(j=0 ; j + N < buffer_len; j++){
/* Window starts at index 0 and is of size N */
printf("\nCurrent window :");
for(s =j; s<j+N; s++){
printf("%d ", buffer[s]);
}
printf("\n");
/* Put K element from N element window */
for(i=0;i<K; i++){
/* Since we wold be doing for every window,
avoiding reallocation of node */
if(max_heap[i]){
max_heap[i]->data = buffer[i+j];
}
else{
max_heap[i] = create_node(buffer[i+j]);
}
}
/* Build min heap with those entered elements */
build_max_heap_ptr(max_heap,K-1);

/*Now for all remaining N-K-1 elements in window,
check if they fit in max heap */
for(i=K+j; i< N+j; i++){
heapNode * root = max_heap[0];
if(buffer[i] < root->data){
root->data = buffer[i];
max_heapify_ptr(max_heap, 0, K-1);
}
}

/*Print the current max heap, it will contain K smallest
element in current window */
printf("K minimum elements in this window :");
for(int x=0; x< K; x++){
printf("%d ", max_heap[x]->data);
}

}
}
/* Driver Program to execute above code */
int main(){
int buffer[10] = {1,4,5,6,3,2,4,8,9,6};

int K= 4;
int N =5;

int size = sizeof(buffer)/ sizeof(buffer[0]);

slide_window(buffer,N, K,size);
return 0;
}
```

Following figures explain how window slides and how heap is updated.
1. Window starts at index 0 and ends at N. We take K minimum elements among N elements and store in max heap. Array is given in below picture with window size of 9 and k = 4.
First step is to create a max heap with first 4 elements of window.

Next we are looking at 4, which is less than max in max heap. So we remove the max from heap and add the new element(4) to heap.

Next is 2, which is less than max in max heap. So we remove the max from heap and add the new element(2) to heap.

Next is 3, which is less than max in max heap. So we remove the max from heap and add the new element(3) to heap.

Next we have 10 and 11 which are greater than root of max heap, so nothing happens.

We come to end of window. Therefore, 4 smallest element in window are [ 1,2,3,4 ]

Next window moves one step ahead, that’s where you discard the max heap and create the new empty one and repeat the process.

We can actually avoid discarding the entire heap when window moves, however complexity of overall algorithm will remain the same. This problem is asked in a different way, which is to find maximum in sliding window.

```#include <iostream>
#include<deque>
using namespace std;

void slidingWindow(int buffer[], int n, int w, int output[])
{
deque<int> Q;
int i;
/*Initilize deque Q for first window, put all W elements, however also
removing elements which cannot be maximum in this window */
for (i = 0; i < w; i++)
{
//This is where we are removing all less than elements
while (!Q.empty() && buffer[i] >= buffer[Q.back()])
Q.pop_back();
// Pushing the index
Q.push_back(i);
}

for (i = w; i < n; i++)
{
output[i-w] = buffer[Q.front()];

//update Q for new window
while (!Q.empty() && buffer[i] >= buffer[Q.back()])
Q.pop_back();

//Pop older element outside window from Q
while (!Q.empty() && Q.front() <= i-w)
Q.pop_front();

//Insert current element in Q
Q.push_back(i);
}
output[n-w] = buffer[Q.front()];
}

int main(){
int a[]={3,5,4,2,-1,4,0,-3};
int n = sizeof(a)/sizeof(a[0]);
int output[n];

slidingWindow(a,n,4,output);
return 0;
}
```

Worst case complexity of sliding window algorithm would be O(n2k). K is included as it takes O(k) complexity to build heap of k elements.

Please share if there is something wrong or missing.

# Kth smallest element in array

Given an array of integers which is non sorted, find kth smallest element in that array. For example: if input array is A = [3,5,1,2,6,9,7], 4th smallest element in array A is 5, because if you sort the array A, it looks like A = [1,2,3,5,6,7,9] and now you can easily see that 4th element is 5.

This problem is commonly asked in Microsoft and Amazon interviews as it has multiple layers and there are some many things that can be tested with this one problem.

## Kth smallest element : Line of thought

First of all, in any interview, try to come up with brute force solution. Brute force solution to find Kth smallest element in array of integers would be to sort the array and return A[k-1] element (K-1 as array is zero base indexed).

What is the complexity of brute force solution? It’s `O(n2)`? Well, we have sort algorithms like merge sort and heap sort which work in `O(n log n)` complexity. Problem with both searches is that they use additional space. Quick sort is another sort algorithm. It has problem that it’s worst case complexity will be `O(n2)`, which happens when input is completely sorted.
In our case, input is given as unsorted already, so we can expect that quick sort will function with `O(n log n)` complexity which is it’s average case complexity. Advantage of using quick sort is that there is no additional space complexity.

Optimising quick sort

Let’s see how quicksort works and see if we can optimize solution further?
Idea behind quicksort is to find the correct place for the selected pivot. Once the pivot is at the correct position, all the elements on the left side of the pivot are smaller and on the right side of the pivot are greater than the pivot. This step is partitioning.

If after partitioning, pivot is at position j, can we say that pivot is actually jth smallest element of the array? What if j is equal to k? Well problem solved, we found the kth smallest element.

If j is less than k, left subarray is less than k, we need to include more elements from right subarray, therefore kth smallest element is in right subarray somewhere. We have already found j smallest elements, all we need to find is k-j elements from right subarray.

What if j is greater than k? In this case, we have to drop some elements from left subarray, so our search space would be left subarray after partition.

Theoretically, this algorithm still has the complexity of `O(n log n)`, but practically, you do not need to sort the entire array before you find k smallest elements.

If you are preparing for a technical interview and need personal coaching along with mock interviews, book a free demo session with us

### Algorithm to find K smallest elements in array

1. Select a pivot and partition the array with pivot at correct position j
2. If position of pivot, j, is equal to k, return A[j].
3. If j is less than k, discard array from start to j, and look for (k-j)th smallest element in right sub array, go to step 1.
4. If j is greater than k, discard array from j to end and look for kth element in left subarray, go to step 1

Let’s take an example and see if this algorithm works? A =  [4, 2, 1, 7, 5, 3, 8, 10, 9, 6 ], and we have to find fifth smallest element in array A.

Start with pivot as first index of array, so pivot = 0, partition the array into two parts around pivot such that all elements on left side of pivot element, i.e. `A[pivot]` are smaller and all elements on right side are greater than `A[pivot]`.

Start with pivot as first index of array, so pivot = 0, partition the array into two parts around pivot such that all elements on left side of pivot element, i.e. `A[pivot]` are smaller and all elements on right side are greater than `A[pivot]`.

In our example, array A will look like below after pivot has found it’s the correct position.

If pivot == k-1 (array is represented as zero base index), then `A[pivot]` is kth smallest element. Since pivot (3) is less than k-1 (4), look for kth smallest element on right side of the pivot.

k remains as it is as opposed to k-j mentioned in algorithm as pivot is given w.r.t entire array and not w.r.t subarray.

In second iteration, pivot = 4 (index and not element). After second execution of quick sort array A will be like

pivot(4) which is equal to k-1(5-1). 5th smallest element in array A is 5.

#### Kth smallest element : Implementation

```package com.company;

/**
* Created by sangar on 30.9.18.
*/
public class KthSmallest {
private void swap(int[] a, int i, int j){
int temp = a[i];
a[i] = a[j];
a[j] = temp;
}
private int partition(int[] a, int start, int end){
int pivot = a[start];
int i  = start+1;
int j  = end;

while(i <= j){
while(a[i] < pivot) i++;
while(a[j] > pivot) j--;

if(i < j) {
swap(a, i, j);
}
}
swap(a, start, j);
return j;
}

public int findKthSmallestElement(int a[], int start,
int end, int k){
if(start <= end){
int p = partition(a, start, end);
if(p == k-1){
return a[p];
}
if(p > k-1)
return findKthSmallestElement(a, start, p, k);
if(p < k-1)
return findKthSmallestElement(a, p+1, end, k);
}
return -1;
}
}
```
```package test;

import com.company.KthSmallest;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertEquals;

/**
* Created by sangar on 28.8.18.
*/
public class KthSmallestTest {

KthSmallest tester = new KthSmallest();
private int[] a = {4, 2, 1, 7, 5, 3, 8, 10, 9};
@Test
public void kthSmallest() {
assertEquals(7, tester.findKthSmallestElement(a,0,8,6));
}

@Test
public void firstSmallest() {
assertEquals(1, tester.findKthSmallestElement(a,0,8,1));
}

@Test
public void lastSmallest() {
assertEquals(10, tester.findKthSmallestElement(a,0,8,9));
}

@Test
public void kGreaterThanSize() {
assertEquals(-1, tester.findKthSmallestElement(a,0,8,15));
}
@Test
public void emptyArray() {
int[] a = {};
assertEquals(-1, tester.findKthSmallestElement(a,0,0,1));
}

@Test
public void nullArray() {
assertEquals(-1, tester.findKthSmallestElement(null,0,0,1));
}
}
```

Complexity of using quicksort algorithm to find the kth smallest element in the array of integers is still O(n log n).

#### Kth smallest element using heaps

Imagine a case where there are a billion integers in the array and you have to find 5 smallest elements from that array. The complexity of O(n log n) is too costly for that use case. Above algorithm using quicksort does not take into consideration disparity between k and n.

We want top k elements, how about we chose those k elements randomly, call it `set A` and then go through all other n-k elements, call it `set B`, check if element from set B (n-k elements) can displace element in set A (k elements)?

What will be the condition for an element from set B to replace an element in set A? Well, if the new element is less than maximum in set A than the maximum in set A cannot be in the set of k smallest elements right?  Maximum element in set A would be replaced by the new element from set B.

Now, the problem is how to quickly find the maximum out of set A. Heap is the best data structure there. What kind of heap: min heap or max heap? Max heap as it store the maximum of the set at the root of it.

Let’s defined concrete steps to find k smallest elements using max heap.

1. Create a max heap of size k from first k elements of array.
2. Scan all elements in array one by one.
1.  If current element is less than max on heap, add current element to heap and heapify.
2. If not, then go to next element.
3. At the end, max heap will contain k smallest elements of array and root will be kth smallest element.

Let’s take an example and see if this algorithm works? The input array is shown below and we have to find the 6th smallest element in this array.

Step 1 : Create a max heap with first 6 elements of array.

Step 2: Take the next element from set B and check if it is less than the root of max heap. In this case, yes it is. Remove the root and insert the new element into max heap.

Step 2: It continues to 10, nothing happens as the new element is greater than the root of max heap. Same for 9.  At 6, again the root of max heap is greater than 6. Remove the root and add 6 to max heap.

Array scan is finished, so just return the root of the max heap, 6 which is the sixth smallest element in given array.

```	public int findKthSmallestElementUsingHeap(int a[], int k){
//https://stackoverflow.com/questions/11003155/change-priorityqueue-to-max-priorityqueue

PriorityQueue<Integer>  maxHeap =
new PriorityQueue<>(k, Collections.reverseOrder());

if(a == null || k > a.length) return -1;
//Create max with first k elements
for(int i=0; i<k; i++){
}

/*Keep updating max heap based on new element
If new element is less than root,
remove root and add new element
*/

for(int i=k; i<a.length; i++){
if(maxHeap.peek() > a[i]){
maxHeap.remove();
}
}
return maxHeap.peek();
}
```

Can you calculate the complexity of above algorithm? `heapify()` has complexity of `log(k)` with k elements on heap. In worst case, we have to do `heapify()` for all elements in array, which is n, so overall complexity of algorithm becomes `O(n log k)`. Also, there is additional space complexity of `O(k)` to store heap.
When is very small as compared to n, this algorithm again depends on the size of array.

We want k smallest elements, if we pick first k elements from a min heap, will it solve the problem? I think so. Create a min heap of n elements in place from the given array, and then pick first k elements.
Creation of heap has complexity of `O(n)`, do more reading on it. All we need to do is delete k times from this heap, each time there will be `heapify()`. It will have complexity of `O(log n)` for n element heap. So, overall complexity would be `O(n + k log n)`.

Depending on what you want to optimize, select correct method to find kth smallest element in array.

Please share if there is something wrong or missing. If you are interested in taking coaching sessions from our experienced teachers, please reach out to us at communications@algorithmsandme.com